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Similar Students , Different Results: 
Why Do Some Schools Do Better? 



EXECUTIVE SUMMARY 

Why do some California elementary schools serving largely low-income students score as much 
as 250 points higher on the state’s academic performance index (API) than other schools with 
very similar students? This study sought answers to that question by surveying principals and 
teachers in 257 California elementary schools serving similar student populations and analyzing 
the results to determine which current K-5 practices and policies are most strongly associated 
with higher levels of student achievement. 

Our study differs from previous effective-schools studies primarily in its scale, standards-based 
content, and targeted yet comprehensive approach. Strong participation rates within schools 
provided extensive data from approximately 5,500 teachers and 257 principals across the state. 

We examined statewide implementation of California’s standards-based reforms, yet focused on 
schools serving large numbers of low-income students. Using the API as our measure, we 
included high-, medium-, and low-scoring schools, which gave us a basis for comparing 
practices. 

The policy context for the study is California’s standards-based accountability system. Many 
experts consider this state’s K-12 academic content standards, adopted in the late 1990s, to be 
among the nation’s most challenging. School APIs are based on student test scores on the 
California Standards Tests, which measure how well students at each school are mastering grade- 
level academic standards. Given this context, we used each school’s most current (2005) API 
score as the primary performance outcome. 

The sample of schools was drawn from the 25th to 35th percentile band of the state’s 2003-04 
School Characteristics Index where student demographic challenge factors are substantial, but not 
the most severe. 

After reviewing the effective schools literature, we developed and field tested the principal and 
teacher surveys, which were designed to explore school qualities, policies, and practices related to 
school success. Specific domains explored were: implementing a coherent, standards-based 
instructional program; involving and supporting parents; using assessment data to improve 
student achievement and instruction; encouraging teacher collaboration and professional 
development; ensuring instructional resources; enforcing high expectations for student behavior; 
and prioritizing student achievement. 

Extensive analysis of the survey findings used regression analysis to determine which activities 
more common at high-performing than at low-performing schools were correlated with higher 
API scores. The practices found to be associated with high performance were: 

• Prioritizing Student Achievement. Where teacher and principal answers to multiple survey 
questions indicated higher expectations for students, their schools had, on average, higher 
API scores than schools whose staffs indicated lower expectations. In more-successful 
schools, both teachers and principals reported that their school has well defined plans for 
instructional improvement and that they put priority on meeting the state’s API goals and the 
federal adequate yearly progress goals. Teachers and principals also report that their schools 
set measurable goals for exceeding the mandated API student subgroup growth targets for 
improved achievement. 

• Implementing a Coherent, Standards-based Curriculum and Instructional Program. 

Teachers who report the following were more likely to be in higher performing schools: 
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schoolwide instructional consistency within grades; curricular alignment from grade-to- 
grade; classroom instruction guided by state academic standards; curriculum materials in 
math and language arts aligned with the state’s standards; in a district that addresses the 
instructional needs of English learners at their school. Principals were more likely to be in 
higher performing schools if they reported that: the district has clear expectations for student 
performance aligned with the district’s adopted curriculum, and the district evaluates the 
principal based on the extent to which instruction in the school aligns with the curriculum. 

• Using Assessment Data to Improve Student Achievement and Instruction. Strongly 
correlated with a higher API was the extensive use of student assessment data by the district 
and the principal in an effort to improve instruction and student learning. For example, 
principals more often reported that they and the district use assessment data from multiple 
sources (curriculum program and other commercial assessments; district-developed 
assessments; the California Standards Tests and the CAT/6) to evaluate teachers’ practices 
and to identify teachers who need instructional improvement. Principals report using this data 
to develop strategies to follow up on the progress of selected students and help them reach 
goals. According to these principals, the district expects all of its schools to improve 
achievement, evaluates principals based on student achievement, and provides support for 
site-level planning related to improving achievement. 

• Ensuring Availability of Instructional Resources. Where more teachers reported having 
regular or standard certification for teaching in California, schools had, on average, higher 
API scores. The same was true of schools where principals more often reported that their 
districts provide sufficient and up-to-date instructional materials as well as support for 
supplementary instruction for struggling students and for facilities management. Teachers 
with at least five years of full time teaching experience were more likely, on average, to be 
from schools with higher APIs. Principal experience was also correlated with higher school 
achievement. 

Besides signaling critical, interrelated practices of more-effective schools, these findings indicate 
that the principal and the district play key roles in school success. Specifically, it appears that: 

• Principal leadership in the context of accountability-driven reform is being redefined to 
focus on effective management of the school improvement process. In general, API scores 
were higher in schools with principals whose responses indicate that they act as managers of 
school improvement, driving the reform process, cultivating the school vision, and 
extensively using student assessment data for a wide variety of school improvement areas of 
focus, including evaluation of teacher practice and assistance to struggling students. 

• District leadership, accountability, and support appear to influence student 
achievement. Principals’ responses indicate that district practices may contribute to a higher 
API in a variety of ways. These include setting clear expectations that schools meet API and 
AYP growth targets, including for subgroups, as well as providing schools with achievement 
data and evaluating principal performance and teacher practices based on that data. They also 
include ensuring: that math and language arts curricula are aligned with state standards; that 
instruction is focused on achievement; that schools have adequate facilities and textbooks as 
well as resources for struggling students. 

Across California, schools serving similar types of student populations can vary widely in how 
well they score on the API. The 257 elementary schools studied were drawn from a fairly narrow 
student demographic band. Yet their 2005 Growth API scores varied by about 250 points. This 
range of scores suggests that while student socioeconomic background is one predictor of 
academic achievement, it is not the sole predictor. What schools do — and what resources they 
have to do it with — can make a difference. With that in mind, the interrelated practices identified 
in this study may help schools in their efforts to improve student achievement. 
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Similar Students , Different Results: 
Why Do Some Schools Do Better? 

A large-scale survey of California elementary schools 
serving low-income students 



In California, the correlation or relationship between school-level student demographics (such as 
percentages of students from families with low incomes and education levels) and school-level 
academic achievement (as measured by the state’s Academic Performance Index, or API) is quite 
high. However, it is also true that two schools serving similarly challenging student populations 
can have very different levels of performance - a difference of as much as 250 points on the API 
(on a scale of 200-1000). Why? This was the central research question we set out to answer. 

By conducting a large scale survey of elementary schools across California serving similarly 
challenged low-income student populations, a collaborative research team (EdSource, Stanford 
University, U.C. Berkeley, and American Institutes for Research) sought to find which current 
K-5 practices and policies are most strongly correlated with high achievement. The study’s 
surveys focused on concrete and actionable practices and policies at the school level, but also 
gathered teacher and principal reports about district and classroom practices. 

Specifically, this study surveyed teachers and principals at California elementary schools in the 
25th to 35th percentile band of the School Characteristics Index (serving high proportions of low- 
income students), then analyzed their survey results against the school’s Academic Performance 
Index score for 2005. 

The collaborative research team is choosing to release its initial findings in October of 2005 
because as elementary schools receive their API growth scores, these findings may be particularly 
helpful in guiding their deliberations around school improvement. Early in 2006 the research 
team will issue a follow-up report with implications from the findings for practice and policy, as 
well as with the results of additional analyses of the survey data and of interviews with 2 1 district 
superintendents with schools in our sample. 



How this study is different 

Over the years, many research studies have examined which practices and policies make schools 
most effective. This study differs in a variety of ways — particularly in terms of its scale, 
standards-based context, and targeted yet comprehensive approach. 

While many studies have examined a group of districts or schools within a region, few have 
examined such a large number of schools located across a broad geography. In total, 257 schools 
from 145 districts throughout California participated. 

Strong participation rates within each school provided extensive data from approximately 5,500 
teachers and 257 principals. All schools in our sample returned the principal survey and the bulk 
of them returned surveys from at least 80% of their K-5 classroom teachers. 

Further, many studies have focused on high-performing schools only, using a variety of measures 
to identify the schools and then examining their practices. By surveying teachers and principals 
from the full range of school API performance — high, middle, and low — this study sheds light on 
what high-performing schools may be doing that low-performing schools are not. 

In addition, the survey questions help identify “intensity ” of practice or policy implementation 
with response scales ranging from weak to strong agreement, or from infrequent to very frequent 
as to how often a reported practice occurs. 



Page 4 • Similar Students, Different Results • EdSource 




Although the study has a large sample and has focused on both high- and low-performing 
schools, it is also targeted — examining a specific organizational and policy context as well as a 
particular student population. In other words — the study looks only at elementary schools 
serving a specific student population (those in the 25th to 35th percentile of the School 
Characteristics Index or SCI) that are operating in California’s current standards-based school 
policy context. The 2004 research review, How Leadership Influences Student Learning by 
Leithwood, Louis, Anderson, and Wahlstrom, recommends targeting studies in this way to gain 
broader knowledge about effective leadership behavior in certain environments. 

In addition, few studies have examined the implementation of standards-based reforms by 
schools across California. While this study was designed mainly to inform schools and districts 
about useful practices and policies related to improving student achievement, its results also 
inform the state of its progress in implementing standard-based reforms. 

Finally, to ensure that the study findings would be most useful to schools and districts, survey 
questions focused primarily on actionable items that can be implemented by other schools, rather 
than on general theories that are less clear in their implications for practice. 



Education Policy Context and Background for California 

In California, as in many other states, standards-based reform is currently the driving force behind 
public education policy. While elements of a standards-based system were in place as early as the 
mid-1980s in California, it was during the mid-1990s that an aligned standards-based education 
system began to develop more significantly. During this period, the state also focused most on 
elementary schools, especially their reading programs. 

The general principle of K-12 standards-based reform is that all elements — curriculum, 
assessments, professional development, financial resources, and accountability systems — are 
aligned to widely agreed-upon, explicit academic content standards set at the state level. The 
standards specifically describe what students should know and be able to do at each grade level. 
Schools’ ability to help students leam the content standards depends in large measure on how 
well the state aligns all of the key elements of the system. 

In 1995 the state initiated a process for the development of California’s academic standards and 
assessments. As of 1999, the State Board of Education (SBE) had adopted statewide academic 
content standards in the four core subject areas — English language arts, mathematics, 
history/social science, and science — with math and English language arts completed first. 

According to the Fordham Foundation’s The State of State Standards 2000 , California adopted 
the most rigorous academic standards in the nation. Fordham gave California an overall grade of 
A-, making it at the time the only state to reach the “A” level for the rigor of its standards. In 
Making Standards Matter 200 1 , the American Federation of Teachers gave California their 
highest ranking for the state’s academic standards in the four core subjects, reporting that the 
state’s standards were "clear and specific." 

The assessment system was established by state law in 1997 and implemented in 1998, but has 
evolved significantly since that time to align with the standards and curriculum. The 
accountability system aligned with the assessment-based system was initially established in 1998- 
99. The Public Schools Accountability Act (PSAA) was passed in the spring of 1999, and the first 
school APIs were issued in November 1999, based on tests from the prior spring. 



Curriculum Frameworks and Textbooks 

In California, the school board in each local district has historically been responsible for 
determining the subjects to teach and how to teach them, within the broad parameters set by the 
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state. As a result of the standards movement, state requirements and recommendations have had 
increasing influence on local choices. 

Today the state specifies several subjects that all California public schools must teach. In 
addition, the State Board of Education approves a curriculum framework for each subject. In each 
of the four core subject areas (math, English language arts, history/social science, science) the 
framework is based on the state-adopted academic standards. The framework document itself 
provides an outline of what should be included in a given course of study and is meant to guide 
school districts and textbook developers. 1 

Finally, the SBE recommends curriculum materials and instructional approaches. For grades K-8, 
the SBE adopts textbooks and other instructional materials for each subject area and each grade 
level. The state gives school districts funds to purchase materials, and a district must choose a 
percentage of its textbooks from the approved list in order to receive those funds. (However, 
districts can request a waiver if they find other materials more appropriate for their schools.) 
While the local school board ultimately decides on its own schools’ textbooks and curriculum, the 
state’s funding of particular textbooks influences those decisions. 

For grades K-6, the SBE selected two curriculum programs for English language arts: Houghton 
Mifflin Reading: A Legacy of Literature and SRA/Open Court Reading. Grades 4-6 can also use 
several other texts. The selections for mathematics were more extensive, with districts having a 
choice of seven programs adopted for the grades K-5 or K-6. 



State Testing System 

California’s assessment system is the Standardized Testing and Reporting or STAR program, 
which the state established in a 1997 law and began implementing in 1998. California public 
schools are required to test all pupils in grades 2-11 unless a parent requests in writing that the 
pupil be exempted. Each summer, the state releases results for testing completed the previous 
spring. The STAR program currently consists primarily of: 

California Standards Tests (CSTs), which are based on the state's academic content standards. 
The CSTs are primarily multiple choice, but for fourth and seventh graders they also include a 
writing test. 

California Achievement Tests, Sixth Edition Survey (CAT/6) which are taken at the same time 
as the CSTs and measure basic skills. Scores indicate the performance of each student relative to 
a national sample of students. Starting in 2005, only third and seventh graders took the CAT/6. 

Currently, the CST serves as the key indicator of school performance in the state’s accountability 
system and the basis for the Academic Performance Index (API) for schools, which was the 
dependent variable or student achievement outcome measure for this study. The State Board of 
Education developed five performance levels for reporting student results on the CSTs: far below 
basic, below basic, basic, proficient, and advanced, with the goal of all students scoring at 
proficient or above. These levels were first applied to the 2001 CSTs in English language arts and 
were integrated into the state’s accountability system beginning in 2001. 

Since 1999, a multiple-choice, norm-referenced test has also been a part of the state’s 
accountability system, but to varying degrees over time. From 1999 through 2002, The Stanford 
Achievement Tests, Ninth Edition (SAT-9) was used, and for the first two years it was the sole 
accountability measure. Beginning in 2003, the SAT-9 was replaced by the California 
Achievement Tests, Sixth Edition (CAT/6) survey form. In 2005, the CAT/6 administration was 
scaled back from grades 2-11 to just grades 3 and 7. Significantly disabled students, who cannot 
take the CAT/6 or CSTs, take the California Alternate Performance Assessment (CAPA). 



'To see California’s curriculum frameworks and other standards-based instructional materials adopted by the SBE, go 

to www.cde.ca.gov/be/st/fr 
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The state conducts two other assessments, the results of which are not included in the API 
calculation. The Spanish Assessment of Basic Education, Second Edition (SAB E/2), is a Spanish- 
language test administered to Spanish-speaking students in grades 2-11 who have been enrolled 
in California schools for less than one year. It covers mathematics and reading and writing in 
Spanish. All students whose primary language is not English take the California English 
Language Development Test (CELDT) when they first enroll in school and each year after until 
school officials determine that they have become English proficient. In contrast to the SABE/2, 
the CELDT evaluates a student’s ability to listen, speak, read, and write in English. 



California’s Accountability System 

Many state policymakers saw an accountability plan as the final ingredient in a standards-based 
education system. Further, in 1994 Title I of the federal Elementary and Secondary Education Act 
(ESEA) called for such a plan. In 1997, the state began to examine how to align accountability 
with the standards, curriculum, and assessments and in 1 999 it enacted the Public School 
Accountability Act (PSAA). Its cornerstone was the Academic Performance Index (API). 2 

An API score is a one-number summary of each spring’s test scores, with different tests receiving 
various weights in the index. With rare exceptions, eveiy school in California is assigned an API 
score. Each school receives an API score between 200 and 1000, which is calculated from student 
scores on the CSTs and CAT/6 (and CAPA if appropriate). In addition to a score for the school as 
a whole, schools receive a score for each “numerically significant” subgroup of pupils 
categorized by ethnicity and poverty 3 . Under the current formula, to be “numerically significant,” 
a subgroup must have 100 students or have 50 or more students that constitute at least 15% of the 
school’s student body. Scores are included for all students who have attended school in the 
district since the prior October, including EL students. 

A school’s API score is used in three different ways. First, schools of the same type (elementary, 
middle, and high schools) are ranked into deciles that each represent 10% of schools. Schools in 
Decile 1 have the lowest scores, and schools in decile 10 have the highest. Since schools’ decile 
rankings indicate how they compare to other schools, there will always be 1 0% of schools in the 
bottom decile — no matter how much the overall scores improve. 

Second, schools are also compared to the 100 schools most like them in terms of student 
background and some other relevant factors. Based on their API score, schools receive a “similar 
school ranking” between one and 10. The similar school ranking indicates how well a school does 
compared to other schools that face a similar level of challenge. The School Characteristics Index 
(SCI), yet another composite index, is used to calculate the similar schools rank. The SCI is made 
up of many student demographic factors and a few school characteristics. All are weighted 
differently, with parent education level receiving the most weight. Multiple linear regressions are 
carried out each year to determine how the index will be calculated for eveiy school in the state. 4 

Finally, the state gives schools a target for improving their API score and tracks the change from 
one year to the next. California’s goal is for every school to have an API score of at least 800. A 
school’s growth target is 5% of the difference between its current score and the state’s goal. 
Schools with API scores of 800 and above need only maintain their score at that level or above 



2 

For a comprehensive overview of how the API is calculated and used, see “Overview of the Academic Performance 
Index School Base Reports 2004” at http://www.cde.ca.gov/ta/ac/ap/documents/overview04b.pdf. 

3 

To date, API subgroups have included: White, Hispanic, African-American, Asian, Filipino, Pacific Islander, American 
Indian/Alaskan, and economically disadvantaged. APIs will be calculated for both Special Education students and English 
learners beginning with the 2005-06 cycle. 

4 

For an in-depth description of how the SCI is constructed, see “Construction of California’s 1999 School 
Characteristics Index and Similar Schools Ranks” at http://www.cde.ca.gov/ta/ac/ap/documents/tdgreport0400.pdf. 
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800. Schools with the lowest API scores have higher growth targets, but they also have more 
room to grow. 

The API system is organized into two-year cycles, with a “Base” score for the first year and a 
“Growth” score for the second year. (The Base and Growth scores can be thought of as “before” 
and “after” snapshots.) Early in the calendar year, each school receives a Base score based on its 
students’ performance on tests from the prior spring. The school is also given growth targets for 
the school as a whole and its subgroups. The Growth score — based on test scores from the 
following spring — is released in the fall. 

Computing the API score for a school (and its subgroups) involves sorting students’ test scores 
into five performance levels. An API score is basically a summary of the distribution of scores 
among the five performance levels, with various subjects and tests receiving various weights in 
that calculation. 

To achieve growth in its API score, a school (or subgroup) needs to have a greater percentage of 
its pupils score in higher performance bands. The API formula rewards growth from the bottom 
of the performance distribution upward more heavily than growth from the middle upward. This 
creates an incentive for a school to work with its lowest-performing students. 



The NCLB accountability system, as implemented in California 

Federal accountability requirements under No Child Left Behind (NCLB) became law in 2002, 
well after California had developed and implemented its accountability system. The state has 
satisfied the federal requirements for accountability under NCLB in part by using elements of its 
own state program. However, NCLB intensifies the focus on the achievement of every subgroup 
of students in a school. Further, the primary measure of success under NCLB is that a 
specific — and gradually increasing — percentage of all subgroups of students scores “proficient” 
or “advanced” on the California Standards Tests in English and math. Schools are also held 
accountable for testing 95% of students in each subgroup and the school as a whole. Failure to do 
so results in the same sanctions that occur if student performance is below expectations. A 
school’s API score must also meet a baseline expectation or improve by one point. Both 
individual schools and school districts as a whole that meet these expectations are said to have 
made AYP — Adequate Yearly Progress. 

Schools and districts that fail to make AYP over two or more consecutive years (and that receive 
funding from the federal Title I program that supports low-income students) face an escalating set 
of consequences — from allowing students to transfer to other schools (with transportation 
provided) to shutting the school down — within a process called Program Improvement. While 
both federal and state accountability programs exact consequences for schools not meeting set 
targets, more California elementary schools are in jeopardy of missing the AYP targets, so this 
federal indicator is now driving many school improvement plans. 

This study’s focus, however, was on how well elementary schools were performing on 
California’s API. We chose to not use a school’s AYP status per se as a dependent variable for 
this study for several reasons. Under AYP, the single “yes” or “no” status question regarding a 
school’s success is an accumulation of multiple data points related to both student test scores and 
participation rates. Failure of any student subgroup to meet the official benchmark on either 
dimension triggers the negative designation. Thus, schools that do not make AYP can be in that 
situation for a host of reasons. Conversely, the threshold for performance is rather low at this 
point, rendering a “yes” on AYP less meaningful as an indicator of a high performing school. The 
next level of analysis, beyond the simple yes or no, would have to look at outcomes for each 
subgroup. That includes far too many variables — and too much variation among schools — to be 
helpful for this study. 
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However, our survey asked questions about the extent to which school staffs prioritize meeting 
their AYP targets as well as their API targets because schools are expected to meet both. 

Schools are addressing the demands of both the state and federal accountability systems by 
focusing on improving student achievement on the California Standards Tests, as these tests form 
the key measure of performance (API and AYP) for both systems. As a result, API scores have a 
strong relationship to the AYP system. 



Research Process and Methods 

The research question — “why do schools serving similarly challenged student populations vary in 
their performance on the API by as many as 250 points?” — drove the research decisions and 
plans. 

Early in the development of this study the research team agreed on several key decisions: 

• The primary performance outcome, or dependent variable, would be the school’s most 
current (2005) API score. 

• The sample of schools would be drawn from a band within the 2003-04 School 
Characteristics Index (25th to 35th percentile) where the student demographic challenge 
factors are substantial, but not the most severe. Limiting the study sample to a narrow SCI 
band helped control for student demographics. 



900 n 



Academic Performance Index (API) scores varied widely among the 
257 elementary schools sampled from a group serving similarly 
disadvantaged students. 
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2003 Student Characteristics Index (SCI)* 
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*25th-35th SCI percentile based on most recent data available in winter 2004, when schools were 
samnled. 



The research team began work early in 2004 by reviewing the research literature on effective 
schools, as well as on high-performing, high-poverty schools. Our initial survey questions for 
principals and for teachers were grounded in seminal research, covering such broad areas as 
School Context, District Role, Core Curriculum, Instructional Strategies, English Learners, 
Kindergarten, Assessment and Data, Professional Development, and Principal Leadership. 

In the spring and summer of 2004 these surveys underwent review by academics who commented 
on their relevance to previous research; by state policymakers to ensure we had captured the 
state’s policies accurately; and by K-12 educators to get feedback on the surveys’ focus and 
wording. The research team’s goal was for the questions to be impartial in their wording and 
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focused on a wide range of potentially meaningful school practices and policies that are 
actionable and likely to be replicable in other schools. 



Field Testing the Surveys 

In the summer of 2004 we piloted the surveys in eleven different elementary schools across the 
state. We asked principals to take the survey before we arrived and then interviewed them about 
their impressions of the instrument, what questions did not make sense, and how on target the 
survey was in capturing what was important to them. We followed that with a general 
conversation with the principal as to what was working or not at the school. These conversations 
led us to some additional survey questions that were not already evident from themes in the 
existing literature. The majority of each school’s K-5 classroom teachers also participated in 
taking the surveys. After they finished, we conducted cognitive interviews to ensure that their 
understanding of the questions matched what we intended to ask. 

As the field-testing progressed, what we heard from school principals and teachers made us 
realize that the research prior to standards adoption in California did not capture all of what 
schools were now doing to raise student achievement in this context. We refined our surveys to 
make sure they reflected the general domains of effective schools practices found in previous 
research but also to capture the specific kinds of classroom, school, and district practices and 
policies we heard were more relevant to the current standards-based education policy 
environment in California. We then shortened the surveys in the hopes of increasing participation. 

The final products were two 45-minute surveys: one for K-5 classroom teachers (46 multi-part 
questions totalling 371 items) and one for elementary principals (36 multi-part questions totaling 
442 items). [To view copies of the surveys go to www.edsource.org] 



The Sampling Process: Selecting and Recruiting the Schools 

[See Appendix A on the study findings page at www.edsource.org for more details] 

The 25th-35th percentile SCI band has approximately 550 elementary schools, most with high 
levels of low-income and English Learner students, many with high percentages of Hispanic 
and/or African-American populations. The research team employed a purposive, stratified 
random sample to choose the schools within this SCI band to approach for participation in the 
study. Our initial goal was to get a sample of 300 schools to participate, including schools from 
eighteen targeted districts across the state as well as all of the API Decile 7 or above (highest 
performing) schools within the band. 

EdSource took on the task of recruiting schools for the study because of its 28 year history of 
strong relationships with districts. 

The sampling plan organized schools into groups (strata) by first creating strata representing a 
small number of specific districts and then classifying the remaining schools into strata defined 
by API decile. These strata were crossed with a stratification according to EdSource subscriber 
status. Because it was anticipated (correctly) that it would be easier to enlist the cooperation of 
schools in EdSource subscriber districts, a higher sampling fraction was specified for these 
districts. Within each stratum, schools were randomly sampled. District offices for sampled 
schools were approached first. A refusal at the district level resulted in all schools in that district 
being dropped. New schools were then randomly sampled and contacted from the districts that 
agreed to cooperate. 

EdSource initially gained the agreement to participate from 269 schools from 154 districts; the 
final sample included 257 schools from 145 districts. 
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Research Methodology and Analyses 

[See Appendix B on the study findings page at www.edsource.org for more details] 

Researchers from each of the four collaborating organizations have met biweekly since March 
2005 to discuss the data analysis, while the technical team has met more frequently to specify 
details of the analyses. Data fde construction was carried out primarily by AIR under the 
direction of Dr. Edward Haertel, the study’s senior technical consultant. 

A very high level of cooperation among the schools in the selected SCI band was obtained, and 
comparisons of participating versus nonparticipating schools suggest that any biases due to less 
than full participation may be safely ignored. To the extent there were differences, the research 
team adjusted for them using sampling weights so that results reported are representative of the 
547 schools in the 25th to 35th percentiles of the SCI. 

Teacher surveys were aggregated to the school level. School-level teacher survey data were then 
merged with principal survey responses as well as demographic and achievement data from 
EdSource and the California Department of Education to create the final school-level analysis 
file. Information from several databases was incorporated in the merged file. 

The next step in the analysis, using the school-level file, was to combine the reliable teacher and 
principal survey items to create composite variables (scales) representing school qualities, 
policies, or practices potentially related to academic success. Within the project — and in this 
document — these were referred to as “sub-domains.” An in-depth analysis of the sub-domains 
was performed to ensure that the groups of items had internal consistency and that the included 
items were positively correlated with achievement. 

The primary achievement outcome of interest was the Academic Performance Index (API). 
Initially, the 2004 "Growth" API was the primary focus of analyses. When the 2005 "Growth" 

API was released in August, analyses were rerun using the 2005 test data and 2005 demographic 
control variables. Results were essentially unchanged; the 2005 results are the primary reporting 
focus. 

As mentioned previously, "Growth" API is a cross-sectional measure based on data from a single 
year. The term "Growth" indicates that it is calculated in the same manner as the preceding year's 
"Base" API. Thus, the arithmetic difference between current-year "Growth" and prior-year "Base" 
APIs represents the change in a school's performance from one year to the next. Only cross- 
sectional measures, not change measures, are used in the analyses for the present study. In 
addition, an outcome variable based on schools' average API over three years — between 2002 and 
2004 — with greater weight on more recent years, was examined. This "API Composite" gave 
results similar to, but generally weaker than, those for recent single -year API outcome variables. 

Results were also examined for school-level variables created by averaging CST scores in 
English language arts across all grades within the school, and similarly for mathematics. Analyses 
of these separate outcome measures showed little difference in variables predictive of 
performance in either subject area. 

The sub-domains were further organized into eight categories ("domains") representing general 
areas of focus like the school's instructional program, parental involvement, use of achievement 
data, and student behavior expectations. For the primary regression analyses, we first regressed 
the outcome (e.g., 2005 Growth API) on all sub-domains within a given domain, together with a 
set of demographic variables chosen to control for residual effects of socioeconomic status and 
student characteristics that were still present within the narrow SCI band specified in framing the 
study population. The primary statistic of interest in examining these regressions was the 
difference in percent of variance accounted for (adjusted R 2 ) for that entire domain and the 
percent of variance accounted for in a regression including demographic variables only. 



Page 11* Similar Students, Different Results • EdSource 



For each domain, we next fit a parsimonious regression model that included school demographic 
characteristics and a subset of the sub-domains that collectively accounted for most of the 
variation in API scores captured using the entire domain. We then used the estimates from this 
regression model to predict the API for each school, but assuming each school had demographic 
characteristics equal to the average of the schools in the sample. The extent to which these 
predicted API scores vary across schools gives us a measure of the extent to which the composite 
variables from each domain account for variation in API scores. A final analysis employed the 
same procedures, but included the final subsets of sub-domains from all eight domains in a single 
model. 

We want to point out that the teacher survey contained a considerable number of questions related 
to school and classroom instructional practices for English learners. This population is 
particularly large in California and research that can shed light on the most effective ways to raise 
their academic achievement is important. However, the research team’s technical group found 
that determining the appropriate dependent variable for studying the responses to this subset of 
survey questions was too complex and methodologically challenging to accomplish within the 
timeframe of this study. We will report descriptive findings related to these survey questions in 
early 2006 (see Further Analyses at the end of this report). 



Descriptive Statistics of the Schools in Our Sample and Their 
Student Populations 

[See Appendix C on the study findings page at www.edsource.org for more details! 



Overall Demographics and Performance of Schools Participating in the 
Study 5 

In 2003-04, when the study’s sample was derived, 547 non-charter elementary schools were in 
the 25th-35th percentile of the school characteristics index (SCI) band. Out of the 269 schools 
identified from this group for the study’s sample, 257 agreed to participate in the study and 
returned the principal’s survey. The overwhelming majority of these schools also returned at least 
80% of the teacher surveys. 

Schools in 145 school districts participated in the study. Fifty-six of these districts had more than 
one school in the study. According to 2004-05 data, the average school enrollment was 588 
students. Of the 257 participating or sample schools: 

• 33% were situated in elementary school districts and the remaining 67% in unified school 
districts. 

• 15% of the schools operated on a year-round calendar. 

• 98% received Title I funding and 34% participated in Program Improvement — meaning that 
these Title I schools had not met the federal Adequate Yearly Progress benchmark for at least 
two consecutive years. 



Data used in this section is from 2004-05 unless otherwise noted. 
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Student Demographics 

The student population attending the participating schools was diverse and faced particular 
challenges. At the median sample school 6 , 40% did not speak English as their primary language 
and 78% participated in the free and reduced-price meals program for low-income families. 
Among all sample schools, the percent of English learners in a school ranged from 1% to 80% 
and the percent of students participating in the free and reduced price-meals program from 1 7% 
to 100%. 

The composition of students by ethnicity at the median school was as follows: 68% were 
Hispanic, 14% white, 4% African American, and 3% Asian. The median value for the 
combination of American Indian, Filipino, Pacific Islander, and multi-ethnic students, along with 
those who did not indicate an ethnicity, was 4%. In most but not all schools the majority 
population was Hispanic. The highest percentage of students of a given ethnicity found in any one 
school was: 99% Hispanic, 94% African American, 54% white, 42% Asian, 38% American 
Indian, 31% Filipino, and 10% Pacific Islander. Conversely, the lowest percentage for most 
ethnic groups was 0%, but all schools had at least 1% Hispanic students. 

In the median sample school, 32% of students had parents who were not high school graduates. 
The median values for other parental education levels were 33% high school graduates, 21% 
some college, and 1 1% completed college or graduate school. 



School Performance 

The 2005 Growth API for the average school participating in the study was 702. The lowest- 
performing school scored 569 and the highest performing school 821. 

The state also assigns a rank from one to 1 0 to each school in the state at the start of each API 
cycle, with a 10 identifying the 10% of schools that are the highest performing. For the 2004 Base 
API, none of the schools in the study were ranked a ten. Only 4% ranked at seven or higher, 
meaning within the top 40% of elementary schools. Conversely, 7% were ranked at one, the 
bottom decile. The remaining 89% of schools ranked from two to six, with the majority a three or 
four. 



Description of Study Surveys and Content Domains for Teachers 
and Principals 

As already described, the survey items were organized into sub-domains that represented general 
categories of school qualities, policies, or practices related to academic success. Those sub- 
domains were then combined into eight more general domains for analysis purposes, as described 
below. Our development of these domains rested on an extensive review of existing research 
related to effective schools, district effectiveness, and standards-based reform. 



Prioritizing Student Achievement 

This domain examined the importance both the school and district placed on setting clear, high, 
and measurable expectations for student achievement. Both teachers and principals were asked 
about the extent to which their school and district communicated high expectations and took 
responsibility for student achievement. Further, they reported the degree of priority given by 
teachers, the principal, and the district to meeting API and AYP targets for subgroups of students 
(such as by race/ethnicity and income level). 



6 The median is found by arranging schools in order of their values on a particular variable and finding the value in the 
middle of the group. As a result, there is no one school that represents the median on all of these factors. 
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Implementing a Coherent, Standards-Based Instructional Program 

This domain contained those sub-domains that most clearly indicated that a school’s curriculum 

and instruction are coherent and aligned with state standards. Those included: 

• The amount of time spent on mathematics and language arts instruction, and the extent to 
which they are protected from interruption and integrated with other subjects. 

• The proportion of teachers in a school who regularly use the same curriculum packages, and 
which ones they used. 

• The extent to which teachers reported alignment and consistency in curriculum and 
instruction, planning and materials. 

• Teacher, principal, and district use of state standards to guide curriculum and instruction. 

• The use of a standards-based report card. 

• The extent to which the district had addressed the needs of English learners (EL). 

Principals were also asked about the extent to which clear district expectations guided curriculum 

coherence; whether they saw themselves as knowledgeable about standards and curriculum; and 

whether the school had implemented a new program for EL students in the last four years. 



Using Assessment Data to Improve Student Achievement and 
Instruction 

This was perhaps the most intensively examined domain in our study in terms of numbers of 
questions asked of both teachers and principals. Under the general topic of data and assessment, 
questions addressed the types of assessment data teachers and principals received, as well as how 
they used this data. We categorized the types of data as follows: 

• CST (California Standards Tests) and CAT/6 assessment data, the state’s standardized tests 
administered each spring. 

• CELDT (California English Language Development Test), an annual assessment of English 
learners. 

• Curriculum program assessments. 

• District-developed assessments. 

• Other commercial assessments. 

• Assessments created by individuals in a school. 

Based primarily on item content — but also on the results of our factor analyses — the sub-domains 
were organized differently for teachers than for principals. Teachers’ responses were organized 
around the frequency with which they reviewed assessment data generally, and the extent to 
which they used the specific data types to monitor student performance and inform their 
instruction. 

The analysis of principals’ responses reflected different questions, including their use of specific 
types of assessments and the extent to which they used each type to monitor achievement, address 
student progress, inform school-wide instructional strategies, and monitor and evaluate the 
practices and performance of teachers. Principals were also asked about the influence of district 
expectations for improving student achievement, and about incentives and activities specifically 
targeted at raising CST and CAT/6 scores. 
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Both principals and teachers also responded to a set of questions regarding the extent to which 
they addressed student achievement by subgroup. 



Ensuring Availability of Instructional Resources 

Given the absence of school-level financial data in California, the study was limited in its 
examination of resources. Data regarding the credentials and experience of educators was 
combined with teachers’ reporting regarding adequate classroom materials, and principals’ 
perceptions of a number of different types of resources, including: 

• The skills, knowledge, and attitudes of the teaching staff at the school. 

• The school’s access to qualified support personnel, supplemental financial resources, and 
supplemental instructional time for students. 

• The extent to which the district provides support for facilities and instructional materials. 

• The amount of regular instructional time, including full-day kindergarten and extended 
school day or year. 



Enforcing High Expectations for Student Behavior 

The examination of this domain was limited to questions regarding the school’s establishment 
and enforcement of policies related to student behavior. Both principals and teachers reported on 
the extent to which the school created an orderly and positive environment for student learning, 
including such areas as attendance policies, enforcement of rules, and respect for cultural 
differences. 



Encouraging Teacher Collaboration and Providing Professional 
Development 

This domain examined three different areas related to the professional environment in the school, 
and looked at a wide range of activities by teachers, principals, and districts. The first area was 
teacher collaboration and professionalism. The sub-domains looked at: 

• The extent to which teachers felt they had influence over schoolwide decisions. 

• The extent to which teachers and principals reported opportunities for teacher collaboration 
around curriculum and instruction, including for EL students. 

• The extent to which teachers and principals perceived that teacher professionalism was 
supported and encouraged within the school and by the district, and the extent to which they 
experienced a continuous learning environment. 

A second strand in this domain related to the development of educator capacity through 
professional development, with sub-domains focused on: 

• The adequacy, influence, and value of a large number of different teacher professional 
development opportunities, including training linked to standards generally, specific 
curriculum programs, instructional strategies, the use of data to inform instruction, and non- 
instructional issues. 

• The frequency of teacher participation in coaching and modeling activities. 

• The extent to which principals gave their district credit for providing teacher professional 
development opportunities. 

• Principals also reported on the extent to which their district provided them with professional 
development, and the value they ascribed to the experiences they had. 
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Finally, several sub-domains explored the principal’s perspective on the hiring, evaluation and 
firing of the teaching staff at his or her school, including: 

• The principal’s perceptions about the district’s success in building and maintaining a strong 
teaching staff. 

• The principal’s capacity to evaluate teacher performance. 

• The principal’s perceptions regarding his or her ability to hire and remove teachers, including 
district and school factors that influence that ability. 



Involving and Supporting Parents 

This domain included sub-domains that looked both at the school’s active engagement with 
parents and its support of parents and families. Teachers were asked about their practices 
involving parents in students’ education; the district’s success in building the community’s 
confidence in the school; and the principal’s relationship with parents. Principal questions were 
more comprehensive, including: 

• The extent to which the school involves parents in students’ education through mechanisms 
such as parent-teacher conferences, school-wide events, and translators for non-English 
speaking parents. 

• The extent to which the school worked to engage parents in schoolwide decisions and 
activities. 

• The extent to which the school provides support services to parents and families, including 
such things as ESL classes, health services, and assistance programs. 



Initial Study Findings 

Within our sample of elementary schools, the “effective schools” domains that proved to be the 
most significant in distinguishing the responses of teachers and principals from the highest 
performing schools from those in the lowest performing were: 

• Prioritizing Student Achievement; 

• Implementing a Coherent, Standards-based Instructional Program; 

• Using Assessment Data to Improve Student Achievement and Instruction; and 

• Ensuring Availability of Instructional Resources. 

Much of what we found in this study was consistent with recent research that has sought to 
understand how standards-based reforms link to improved instruction and thus to positive student 
outcomes. Perhaps the central message is that no single action, or even category of actions, can 
alone provide a clear advantage related to student performance. 

Education researchers such as Smith and O’Day (1991) have examined the impact of reforms on 
student achievement, arguing that instructional improvement is unlikely to result from a single 
policy or practice. As researchers from the Consortium for Policy Research in Education (CPRE) 7 
note, “schooling consists of complex processes, and it would be remarkable if there was only one 
best way to improve it.” 

This study’s findings are consistent with that theory. Our findings appear to indicate that the 
cumulative effect of the state’s policies on school practices differs among schools that serve low- 
income children. We have also identified at least some of the attitudes and activities that set the 



7 Cohen, D.K. and Hill, H.C. Learning Policy. New Haven, CT: Yale University. 2001 
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higher performing schools apart, such as how well or frequently activities are implemented, 
including high expectations for student performance, curriculum coherence, and the use of 
assessment data for school improvement management. 

Additionally, we found that schools that tended to demonstrate a strong culture of high 
expectations — as evidenced by their attention to meeting and exceeding state and federal 
accountability targets and setting high standards for student achievement — on average had higher 
API scores. Abelmann and Elmore (1999), among others, describe a similar alignment between 
schools’ internal accountability mechanisms (such as prioitizing student achievement) and 
external accountability requirements (such as meeting API/AYP targets). 



Interpreting the Findings 

This study’s findings represent the results of a regression analysis — a tool that uses statistical 
techniques to identify correlations among variables in large databases. The regression analysis 
allowed the researchers to isolate separate distinct correlations between schools' API scores and 
several related independent variables thought to explain variation in API scores. This study does 
not prove that the four domains that are correlated with higher API scores have actually caused 
the higher API scores. Rather, it indicates that schools that report more strongly that they have 
implemented more of the practices included in each of the four domains have, on average, higher 
API scores than schools that report fewer of the practices. 

To quantify the magnitude of API effects that might be attributable to the various domains we 
studied, we estimated the expected differences in API scores associated with changes in teacher, 
principal, and district engagement across the kinds of activities described. Sets of practices and 
policies were grouped together for this analysis and it is the predicted effect of the combined 
practices — not any single practice — that is reflected. The table below presents these domain- 
specific API effects. Because the practices associated with different domains tend to occur 
together (i.e., schools high on one domain tend to be high on others), these effects are not 
additive . Therefore, the API score difference associated with two or more domains of practices 
will be substantially less than the sum of the separate effects for the same domains. 

Magnitude of API Differences 

(holds demographics constant at sample mean) 



Domain 


Standard Deviation of 
Predicted API Distribution 


Implementing a Coherent, Standards-Based Instructional 
Program 


17.6 


Ensuring Availability of Instructional Resources 


16.9 


Using Assessment Data to Improve Student Achievement and 
Instruction 


16.7 


Prioritizing Student Achievement 


16.3 


Enforcing High Expectations for Student Behavior 


12.3 


Encouraging Teacher Collaboration and Professional 
Development 


11.0 


Involving and Supporting Parents 


9.9 



As these data indicate, our findings did not show that practices to strengthen teacher collaboration 
and professional development, enforce high expectations for student behavior, or increase 
involvement and support of parents were unimportant in terms of contributing to a school’s API. 
They were not, however, nearly as strong as the other four domains in terms of differentiating the 
lowest performing schools from the highest in our sample of California elementary schools with 
high percentages of low income students. 
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The Findings 

Prioritizing Student Achievement 

Elementary schools where teachers and principals answered multiple survey questions in ways 
that indicated their higher expectations for students had, on average, higher API scores than 
similar schools with lower reported expectations. For example, when teachers reported that their 
school has a vision focused on student learning outcomes, their schools were more likely to be 
high scoring. The same was true of principal responses on similar questions. And when teachers 
and principals report that the principal communicates a clear vision for the school, sets high 
standards for student learning, and makes expectations clear to teachers for meeting academic 
achievement goals, the school is more likely to be high achieving. Equally important, better 
school performance seems to be associated with both teachers’ and principals’ reports that 
teachers at the school take responsibility for and are committed to improving student 
achievement. 

A shared culture within the school regarding the value of improving student achievement and a 
sense of shared responsibility for it seems to separate higher performing schools in our sample 
from those with lower APIs. But beyond “values,” both teachers and principals reported that 
their school has well defined plans for instructional improvement, and that they make meeting the 
state’s API goals and the NCLB adequate yearly progress goals a priority. Both teachers and 
principals report that their schools set measurable goals for exceeding the mandated API student 
subgroup growth targets for improved achievement. Principals at higher performing schools also 
report on average that their school’s statewide rank and similar schools ranking on the API 
influence school wide instructional priorities, and that they are clear about their district’s 
expectations for meeting API and AYP growth and subgroup targets. 



Implementing a Coherent, Standards-based Curriculum and Instructional Program 

Our findings are consistent with previous research on the value of a coherent curricular program. 
Teachers who report schoolwide instructional consistency within grades — and curricular 
alignment from grade-to-grade — work in schools that performed better on average. Examples of 
practices teachers report using to accomplish this coherency include examining the scope or 
sequence of curriculum topics and reviewing a grade-level pacing calendar. 

Those teachers who reported that their school has identified essential standards and that their 
classroom instruction is guided by state academic standards were also more likely to be in high 
performing schools. They report that the school’s curriculum materials in math and language arts 
are aligned with the state’s standards and that they frequently map state curriculum standards onto 
their classroom lesson plans. Teachers at higher performing schools also more often report that 
their district addresses the instructional needs of English language learners at their school. 

Principals who report a strong district role in this domain are also from higher performing 
elementary schools. These principals say the district has a coherent grade-by-grade curriculum 
that it uses for all schools and that the district expects the principal to ensure implementation of 
the curriculum. These principals report that the district has clear expectations for student 
performance aligned with the district’s adopted curriculum and that it evaluates the principal 
based on the extent to which instruction in the school aligns with the curriculum. 



Using Assessment Data to Improve Student Achievement and Instruction 

Another practice strongly correlated with a higher API among our sample of elementary schools 
was the extensive use of student assessment data by the district and the principal in an effort to 
improve instruction and student learning. As an example, principals from better performing 



Page 18* Similar Students, Different Results • EdSource 




schools more often reported that they and the district use assessment data from multiple sources 
— curriculum program and other commercial assessments, district-developed assessments, and 
the California Standards Tests and the CAT/6 — to evaluate teachers’ practices and to identify 
teachers who need instructional improvement. Principals who reported frequently and personally 
using assessment data to address the academic needs of students in their schools led, on average, 
higher performing schools. They report using this data to develop strategies to help selected 
students reach goals and to follow up on the progress of selected students. In addition, they 
review this data both independently and with individual teachers. 

These same principals report a clear understanding of their district’s expectations for improving 
student achievement, which may help motivate and support them. The principals report that their 
districts expect that all schools in the district will improve student achievement and evaluate 
principals based upon student achievement. The principals report that the district also provides 
support for site-level planning related to improving achievement. 

In schools where assessment data from the California Standards Tests and the CAT/6 influence 
schoolwide attention to improving student achievement, the API also tends to be higher. Teachers 
report receiving CST/CAT-6 test data in a variety of formats: for all students in their grade level; 
disaggregated by specific skills for all students in their classrooms; and disaggregated by student 
subgroup for students in their classrooms. Principals report using the CST and CAT/6 data to 
examine school-wide instructional issues, to develop strategies for moving students from below 
basic and basic to proficient, to compare grades within the school, to identify struggling students 
and evaluate their progress, and to inform and communicate with parents. 



Ensuring Availability of Instructional Resources 

API performance was higher in schools where principals reported that a higher proportion of their 
teaching staff had the following qualities (listed roughly in descending order of importance): 

• demonstrated ability to raise student achievement 

• strong content knowledge 

• good fit with the school culture 

• training in curriculum programs 

• ability to map curriculum standards to instruction 

• supportive of colleague’s learning and improvement 

• able to use data from student assessments 

• familiar with the school community 

• excited about teaching 

• familiar with state standards 



The schools where more teachers reported having regular or standard certificates for California 
also had, on average, higher API scores. 

The principals who more often reported certain district practices were also more likely to be from 
high performing schools. They said their districts ensure the school has: up-to-date instructional 
materials, support to provide supplementary instruction for struggling students, enough 
instructional materials for all students, and support for facilities management. The same was true 
for teachers who tended to confirm these perceptions, reporting that every student in their 
classroom has a copy of the current version of the textbook in language arts and math and that the 
principal ensures the teachers have adequate classroom materials. The principals also reported a 
strong understanding of what their district expected from them in terms of facilities upkeep and 
management. 

Years of educator experience also mattered, but less so than these other items. Teachers who were 
more likely to have at least five years of full time teaching experience were, on average, from 
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schools with higher APIs. Principal years of experience was also correlated with higher school 
achievement. 



Another View: The Role of the Principal and the District 

Principal leadership in the context of accountability-driven reform is being redefined to 
focus on effective management of the school improvement process. 

In general, API scores were higher in schools with principals whose responses indicate that they 
act as managers of school improvement, driving the reform process and cultivating the school 
vision. In particular, they were more likely to embrace the state’s academic standards and to 
ensure classroom instruction was based on them. They prioritized meeting and even exceeding 
state and federal accountability targets for school performance. In addition, they reported 
personally and extensively using student data for instructional management purposes — not only 
to evaluate the progress of students, but also to examine school-wide and teacher practices, 
develop strategies to help selected students reach goals, and identify teachers who need 
instructional improvement. Finally, they ensured that teachers and students had adequate texts 
and classroom materials. 



District leadership, accountability, and support appear to influence student achievement. 

Another theme interwoven in the study’s data was how the district may influence school 
performance. While the study was not designed to examine the influence of district policies per 
se, principals at high-performing schools tended to perceive many aspects of the district role 
differently from principals at low-performing schools. Based upon principals’ survey responses, it 
appears that districts may contribute to a higher API at these elementary schools in a variety of 
ways. Specifically, principals at high-performing schools said their districts were clear in their 
expectations that schools meet both the API and AYP growth targets for the school as a whole 
and for subgroups. They ensured that the school curricula in math and language arts were aligned 
with state standards and that instruction was focused on student achievement. In addition, these 
districts ensured that schools had adequate facilities, textbooks, and resources for struggling 
students. They also provided schools with student achievement data and evaluated principal 
performance and teacher practices based on that data. 



Further Analyses 

The collaborative research team is choosing to release its initial findings in October of 2005 
because as schools in the 25th to 35th SCI band receive their API growth scores, these findings 
may be particularly helpful in guiding their deliberations around school improvement. Additional 
analyses will be reported in early 2006. 

Our survey findings point to a strong district role in providing leadership, direction, and support 
to improving schools. As part of this study the researchers interviewed 21 superintendents or 
associate/sub superintendents in 17 different districts that had high-performing schools in our 
sample. We asked them to talk about the strategies they had used to foster improvements in 
school performance and student achievement. We plan to report on their observations in early 
2006. 

In addition, the principal survey in this study included an open-ended question, asking principals 
to identify the three most effective things their schools had done to improve student achievement. 
We hope to analyze these responses by the school’s 2005 API, to see how those strategies varied 
and report our findings in early 2006. 
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High API scores were also correlated with teachers reporting frequent use of SRA/Open Court 
Reading for reading instruction and Scott Foresman CA Mathematics for math instruction. These 
teachers were also more likely to engage in the various effective practices described above. 
However, this study was not designed in a way that could determine the effects of specific 
curriculum programs independent of other schooling practices. We plan to ran additional analyses 
to test the robustness of these results and report on them in early 2006. 

The teacher survey in our study included a fairly extensive set of questions around instructional 
practices employed for English learner students. We plan to report the descriptive statistics from 
this part of the survey in early 2006; if additional funding is secured, we will also analyze these 
survey findings against appropriate dependent outcome measures to explore which practices seem 
to be working best. 

Other analyses are possible with this rich and unique data set. 

In early 2006 we will issue a follow-up report(s) to this study which will include the results of 
analyses identified above, a connection of the report’s major findings to previous education 
research, and more discussion of the state policy and local school implications of all the findings. 

For a copy of this report, more information related to the study, the appendices mentioned in 
various sections, and a full bibliography including the works cited, go to www.edsource.org and 
click on the homepage link to the Study. 



Conclusion 

Across California, schools serving similar types of student populations can vary widely in how 
well they score on their API. The 257 elementary schools (serving 135,673 K-5 students) that this 
research team studied were drawn from a fairly narrow band in terms of student demographics 
(percent low income, English learner, ethnic/racial subgroups). Yet their 2005 Growth API scores 
varied by as much as 252 points. 

A school’s API score reflects how well its students are performing on the annual California 
Standards Tests. This one test is limited: it is not the only way for a school to measure how well 
their students are mastering the rigorous academic content of the state’s grade -by-grade 
standards; and it also does not measure the other important things that elementary students may 
be learning at school — about art and science and music, about citizenship and tolerance of 
differences, about themselves and their sense of place in the world. 

But an elementary school’s API score provides the state and the public with a consistent and easy 
way to grasp information about the progress its students are making toward mastering the 
important math and reading and writing skills that will enable them to succeed academically in 
later grades. 

For this reason, among others related to accountability, a school’s API score represents an 
important measure of student learning. 

The range of API scores in our sample suggests that while the socioeconomic backgrounds of 
students is one predictor of academic achievement, it is not the sole predictor. What schools 
do — and what resources they have to do it with — can make a difference. With that in mind, the 
interrelated practices identified in this study may help schools in their efforts to improve student 
achievement. 
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INSTRUCTIONS 

□ When marking your responses on this survey, please use a number 2 pencil (preferable) or a 
ballpoint pen with dark ink. Do not use a felt tip pen or a Sharpie marker. 

□ Please indicate your answers by either checking {■/) or making an “X ” in the appropriate 
boxes. Write clearly and distinctly, avoiding any extraneous marks on the page. 

□ If you would like to change an answer, please erase any pencil marking completely or 
clearly strike through a mark made with a pen. 

□ For items that ask you to fill in a written response, please print your response in capital 
letters using one letter per box provided. 

When you have completed the survey, please return it to AIR in the enclosed envelope. 

Thank you very much for taking the time to complete this survey! 



H Improving Achievement for Low-Income Students: What Makes a Difference? H 



SECTION I: School Context 

1. How much do you agree or disagree with each of the following statements? 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. 


Our school has a clear vision that is focused on student 
learning outcomes 


□ 


□ 


□ 


□ 


b. 


Our school has well defined plans for instructional 
improvement 


□ 


□ 


□ 


□ 


c. 


Our school assesses the effectiveness of our plans for 
instructional improvement 


□ 


□ 


□ 


□ 


2. 


How much do you agree or disagree with each of the following statements? 






Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. 


At our school, there is an enforced attendance policy 


□ 


□ 


□ 


□ 


b. 


At our school, there is an enforced dress code for 
students 


□ 


□ 


□ 


□ 


c. 


At our school, there are enforced rules for student 
behavior 


□ 


□ 


□ 


□ 


d. 


At our school, there is a zero tolerance policy toward 
bullying 


□ 


□ 


□ 


□ 


e. 


At our school, students respect cultural differences 


□ 


□ 


□ 


□ 


3. 


How much do you agree or disagree with each of the following statements? 






Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. 


Classroom instruction is guided by state standards 


□ 


□ 


□ 


□ 


b. 


Our school has identified essential/key standards 


□ 


□ 


□ 


□ 


c. 


Our school uses a standards-based report card 


□ 


□ 


□ 


□ 


d. 


Meeting our API growth target is a priority 


□ 


□ 


□ 


□ 


e. 


Meeting our API subgroup (e.g., racial/ethnic, EL) 
targets is a priority 


□ 


□ 


□ 


□ 


f. 


Meeting our AFP subgroup (e.g., racial/ethnic, EL) 
targets is a priority 


□ 


□ 


□ 


□ 


g- 


Our school sets measurable goals for EXCEEDING the 
mandated API subgroup growth targets 


□ 


□ 


□ 


□ 



■ BIKJW 



Page 1 



H Improving Achievement for Low-Income Students: What Makes a Difference? H 



4. How much do you agree or disagree with each of the following statements? 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. 


Teachers take responsibility for student achievement 


□ 


□ 


□ 


□ 


b. 


Teachers are committed to improving student 
achievement 


□ 


□ 


□ 


□ 


c. 


Teachers communicate to students that education is 
important 


□ 


□ 


□ 


□ 


d. 


Teachers provide support to struggling teachers 


□ 


□ 


□ 


□ 


e. 


Teachers are involved in making important decisions at 
this school 


□ 


□ 


□ 


□ 


f. 


Teachers discuss assumptions about race and student 
achievement 


□ 


□ 


□ 


□ 



5. How frequently do the following parent activities occur at your school? 





Once or 
More 
a Week 


Once or 
Twice 
a Month 


A Few 
Times 
a Year 


Once 
a Year 


Never 


a. 


Parent-teacher conferences 


□ 


□ 


□ 


□ 


□ 


b. 


Special subject area events (e.g., science fair, 
art show) 


□ 


□ 


□ 


□ 


□ 


c. 


Parents provide instructional support in 
classrooms 


□ 


□ 


□ 


□ 


□ 


d. 


Workshops or courses for parents 


□ 


□ 


□ 


□ 


□ 


e. 


Parents involved in governance issues 


□ 


□ 


□ 


□ 


□ 


f. 


Parents involved in budget decisions 


□ 


□ 


□ 


□ 


□ 
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6. Does your school have any of the following to facilitate parent participation? 





Yes 


No 


a. 


Staff assigned to maintain parental involvement 


□ 


□ 


b. 


A log of parent participation maintained by parents or staff 


□ 


□ 


c. 


A reliable system of communication with parents (e.g., newsletter, 
phone tree) 


□ 


□ 


d. 


Services to support parent participation (e.g., child care on site, 
transportation) 


□ 


□ 


e. 


Open houses 


□ 


□ 


f. 


A parent drop-in center or lounge 


□ 


□ 


g- 


A translator for parents at school meetings 


□ 


□ 


h. 


A translator for parents at parent-teacher conferences 


□ 


□ 


i. 


Group meetings with parents to explain academic expectations 


□ 


□ 


j- 


Support groups targeted for parents by racial, ethnic, or linguistic 
backgrounds 


□ 


□ 



7. For each of the following services, please indicate if it is available for every student who 
wants the service, for some students who want the service, or if the service is not provided. 





For 

Every 

Student 


For 

Some 

Students 


Service 

Not 

Provided 


a. 


A pre-school program 


□ 


□ 


□ 


b. 


A half-day kindergarten program 


□ 


□ 


□ 


c. 


A full-day kindergarten program 


□ 


□ 


□ 


d. 


An after-school academic program 


□ 


□ 


□ 


e. 


A summer school/intersession academic program 


□ 


□ 


□ 


f. 


A breakfast program for students 


□ 


□ 


□ 


g- 


A food/clothing assistance program 


□ 


□ 


□ 



■ DHULH 
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8. Does your school have access to any of the following resources? 





Dedicated to 
our 
School 


Shared 

Through 

District 


Not 

Available 


a. Curriculum specialist 


□ □ □ 


b. School coach (e.g., school-wide reform) 


□ □ □ 


c. Principal coach 


□ □ □ 


d. Social worker 


□ □ □ 


e. School nurse 


□ □ □ 


f. Psychological counselor 


□ □ □ 


g. Parent school liaison 


□ □ □ 


h. English Language (EL) Learner specialist 


□ □ □ 


i. Special Education specialist 


□ □ □ 


j . Equity /Diversity Coach 


□ □ □ 


k. Truancy officer 


□ □ □ 


1. ESL classes for parents 


□ □ □ 


m. Parenting skills classes 


□ □ □ 


n. Conflict resolution program or services 


□ □ □ 


o. Health services for families 


□ □ □ 



■ EMTIX 
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9. In the last four years, has your school done any of the following to improve student 
achievement? 







Yes 


No 


Don?t 

Know 


a. 


Transferred teachers to different grades or subjects 


□ 


□ 


□ 


b. 


Implemented a school plan that addressed performance on CAT-6/CSTs 


□ 


□ 


□ 


c. 


Implemented a school plan that addressed gaps in student achievement 


□ 


□ 


□ 


d. 


Increased time for test preparation activities 


□ 


□ 


□ 


e. 


Increased the length of the school day 


□ 


□ 


□ 


f. 


Increased the length of the school year 


□ 


□ 


□ 


g- 


Increased the amount of professional development offered 


□ 


□ 


□ 


h. 


Increased the amount of teacher collaboration time 


□ 


□ 


□ 


i. 


Implemented a new program for English Learners 


□ 


□ 


□ 


j- 


Used incentives for students related to performance on CAT-6/CSTs 


□ 


□ 


□ 


k. 


Provided release time for teachers to prepare for CAT-6/CSTs 


□ 


□ 


□ 


1. 


Worked with a whole-school reform provider (e.g., America?s Choice, 
Different Ways of Knowing) 


□ 


□ 


□ 


m. 


Implemented an alternative school instructional model (e.g., project- 
based learning, parent-participation school) 


□ 


□ 


□ 



■ FSMTU 
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SECTION II: Principal Leadership 
10. How frequently do you do each of the following? 





Once or 
More 
a Week 


Once or 
Twice 
a 

Month 


A Few 
Times 
a Year 


Once 
a Year 


Never 


a. 


Participate in a grade-level meeting 


□ 


□ 


□ 


□ 


□ 


b. 


Formally evaluate teachers 


□ 


□ 


□ 


□ 


□ 


c. 


Conduct walkthroughs 


□ 


□ 


□ 


□ 


□ 


d. 


Teach a demonstration/model class 


□ 


□ 


□ 


□ 


□ 



11. Given your daily demands, what priority do you place on each of the following 
responsibilities? 







High 


Moderate 


Low 


Not a 






Priority 


Priority 


Priority 


Priority 


a. 


Communicate a clear vision for our school 


□ 


□ 


□ 


□ 


b. 


Make expectations clear to teachers for meeting 
academic achievement goals 


□ 


□ 


□ 


□ 


c. 


Act as a knowledgeable source concerning standards 
and curriculum 


□ 


□ 


□ 


□ 


d. 


Set high standards for student learning 


□ 


□ 


□ 


□ 


e. 


Provide support for classroom discipline and order 


□ 


□ 


□ 


□ 


f. 


Ensure that teachers have time for planning 


□ 


□ 


□ 


□ 


g- 


Praise and recognize teachers 


□ 


□ 


□ 


□ 


h. 


Encourage teachers to take a leadership role in our 
school 


□ 


□ 


□ 


□ 


i. 


Provide teachers with adequate classroom materials 


□ 


□ 


□ 


□ 


j- 


Ensure that teachers receive adequate professional 
development to improve instruction 


□ 


□ 


□ 


□ 


k. 


Secure additional resources from outside sources (e.g., 
state/federal grants, grants from private sources and/or 
foundations) 


□ 


□ 


□ 


□ 


1 . 


Secure additional resources from our district 


□ 


□ 


□ 


□ 


m. 


Build strong relationships with parents 


□ 


□ 


□ 


□ 
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12. To what extent do the following influence your school-wide instructional priorities ? 





A 

Great 

Extent 


A 

Moderate 

Extent 


A 

Minor 

Extent 


Not at 
All 


a. Results from the CST/CAT-6 


□ 


□ 


□ 


□ 


b. Results from the CELDT 


□ 


□ 


□ 


□ 


c. Benchmark assessments conducted by teachers 


□ 


□ 


□ 


□ 


d. Curriculum program test results 


□ 


□ 


□ 


□ 


e. Student grades and report cards 


□ 


□ 


□ 


□ 


f. Your own classroom observations 


□ 


□ 


□ 


□ 


g. School statewide ranking on API 


□ 


□ 


□ 


□ 


h. Similar school ranking on API 


□ 


□ 


□ 


□ 


i. Requirements that come from private grant funding 


□ 


□ 


□ 


□ 


j . Requirements that come from state-funded school 
improvement programs (e.g., II/USP, HPSGP) 


□ 


□ 


□ 


□ 


k. Requirements that come from other categorical 
funding programs 


□ 


□ 


□ 


□ 


1. Research on best instructional practices 


□ 


□ 


□ 


□ 


m. Practices found to be successful in high performing 
schools in my district 


□ 


□ 


□ 


□ 


n. Practices found to be successful in high performing 
schools outside my district 


□ 


□ 


□ 


□ 
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13. How much do you agree or disagree with each of the following statements? 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. I have adequate time to conduct teacher evaluations 


□ □ □ □ 


b. I use teacher evaluations to address professional 
performance 


□ □ □ □ 


c. I am able to give curricular issues the attention they 
deserve 


□ □ □ □ 


d. I ensure that teachers use adopted curriculum 
packages 


□ □ □ □ 


e. I make it difficult for ineffective teachers to stay in 
my school 


□ □ □ □ 



14. Are you able to hire teachers of your own choosing? 

Yes 

No If No, Go to Question 16 on the next page. 

15. How have the following factors affected your ability to hire teachers of your choice? 

(N/A=Not Applicable) 





Helped 


No 

Effect 


Hindered 


N/A 


a. 


District office policies and practices 


□ 


□ 


□ 


□ 


b. 


Prescreening of candidates by district office 


□ 


□ 


□ 


□ 


c. 


Teacher assignment as specified in teacher union 
contracts 


□ 


□ 


□ 


□ 


d. 


Level of salary and benefits 


□ 


□ 


□ 


□ 


e. 


Overall quality of the applicant pool 


□ 


□ 


□ 


□ 


f. 


Level of student achievement at your school 


□ 


□ 


□ 


□ 


g- 


School reputation 


□ 


□ 


□ 


□ 


h. 


Student demographics 


□ 


□ 


□ 


□ 


i. 


The number of EL students at your school 


□ 


□ 


□ 


□ 


j- 


The neighborhood in which your school is located 


□ 


□ 


□ 


□ 


k. 


The quality of school facilities 


□ 


□ 


□ 


□ 



■ IABHG 
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16. What proportion of your current teaching staff has the following characteristics? 





All of 
Staff 


Most of 
Staff 


About 
Half of 
Staff 


Some 
of Staff 


None 
of Staff 


a. 


Training in our curriculum program(s) 


□ 


□ 


□ 


□ 


□ 


b. 


The ability to speak the home language of EL 
students 


□ 


□ 


□ 


□ 


□ 


c. 


The ability to use data from student assessments 


□ 


□ 


□ 


□ 


□ 


d. 


Familiarity with California state standards 


□ 


□ 


□ 


□ 


□ 


e. 


Familiarity with the school community 


□ 


□ 


□ 


□ 


□ 


f. 


Being excited about teaching 


□ 


□ 


□ 


□ 


□ 


g- 


Fitting well into your school culture 


□ 


□ 


□ 


□ 


□ 


h. 


A demonstrated ability to raise student 
achievement 


□ 


□ 


□ 


□ 


□ 


i. 


Strong content knowledge 


□ 


□ 


□ 


□ 


□ 


j- 


The ability to map curriculum standards to 
instruction 


□ 


□ 


□ 


□ 


□ 


k. 


Struggling in the classroom 


□ 


□ 


□ 


□ 


□ 


1 . 


Likely to leave teaching in the next year or two 


□ 


□ 


□ 


□ 


□ 


m. 


Support colleague?s learning and improvement 


□ 


□ 


□ 


□ 


□ 



17. In the last 4 years, as principal of this school, how many teachers have you: (If you have 
been the principal of this school fewer than 4 years, please answer about the time you have been 
principal of this school. If none, enter “00”.) 



a. Wanted to remove? 

b. Attempted to remove? 

c. Successfully removed? 



Teacher(s) 

Teacher(s) 

Teacher(s) 



■ JRSAE 
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18. How have the following factors affected your ability to remove unsatisfactory teachers 
from your school? (N/A=Not Applicable) 





Helped 


No 

Effect 


Hindered 


N/A 


a. 


District office policies and practices 


□ 


□ 


□ 


□ 


b. 


Prescreening of candidates by district office 


□ 


□ 


□ 


□ 


c. 


Teacher assignment as specified in teacher union 
contracts 


□ 


□ 


□ 


□ 


d. 


Level of salary and benefits 


□ 


□ 


□ 


□ 


e. 


Overall quality of the applicant pool 


□ 


□ 


□ 


□ 


f. 


Level of student achievement at your school 


□ 


□ 


□ 


□ 


g- 


School reputation 


□ 


□ 


□ 


□ 



19. Has your school received additional funding other than state/district funds in the last 4 
years? 

Yes 

No If No, Go to Question 21 on the next page. 

20. How effective was additional funding in helping your school improve student achievement? 

(N/A=Not Applicable) 







Very 

Effective 


Somewhat 

Effective 


Not 

Effective 


N/A 


a. 


State-funded school improvement programs (e.g., 
II/USP, HPSGP) 


□ 


□ 


□ 


□ 


b. 


Private foundation grant(s) 


□ 


□ 


□ 


□ 


c. 


Corporate grant(s) 


□ 


□ 


□ 


□ 


d. 


In-kind donations 


□ 


□ 


□ 


□ 


e. 


Donations from parents/community members 


□ 


□ 


□ 


□ 


f. 


Donations from school/district foundations 


□ 


□ 


□ 


□ 


g- 


Other (please specify): 


□ 


□ 


□ 


□ 
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SECTION III: The Role of the District 

21. I understand my district?s expectations in regard to the following: 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. Implementation of our Language Arts curriculum 
program 


□ 


□ 


□ 


□ 


b. Implementation of our Math curriculum program 


□ 


□ 


□ 


□ 


c. Student homework policies 


□ 


□ 


□ 


□ 


d. Student discipline procedures 


□ 


□ 


□ 


□ 


e. Teacher evaluation process 


□ 


□ 


□ 


□ 


f. Parental involvement/support 


□ 


□ 


□ 


□ 


g. Facilities upkeep/conditions 


□ 


□ 


□ 


□ 


h. Financial management 


□ 


□ 


□ 


□ 


i. My performance as a school leader 


□ 


□ 


□ 


□ 


j . Student achievement on state standards 


□ 


□ 


□ 


□ 


k. Meeting our API growth target 


□ 


□ 


□ 


□ 


1. Meeting our API subgroup target(s) 


□ 


□ 


□ 


□ 


m. Meeting our AYP subgroup target(s) 


□ 


□ 


□ 


□ 



22. How would you characterize the support your district provides to your school in the 
following areas? 







Excellent 


Good 


Fair 


Poor 


a. 


Professional development for teachers focusing on 
curriculum 


□ 


□ 


□ 


□ 


b. 


Professional development for teachers focusing on 
multicultural/diversity issues 


□ 


□ 


□ 


□ 


c. 


Professional development for teachers focusing on 
classroom management/student behavior 


□ 


□ 


□ 


□ 


d. 


Professional development focusing on English Learners 


□ 


□ 


□ 


□ 


e. 


Professional development for you as a principal 


□ 


□ 


□ 


□ 


f. 


Assistance communicating with parents regarding academic 
expectations 


□ 


□ 


□ 


□ 


g- 


Facilities management 


□ 


□ 


□ 


□ 


h. 


Site-level planning related to improving achievement 


□ 


□ 


□ 


□ 


i. 


Resources for supplementary instruction for struggling 
students 


□ 


□ 


□ 


□ 



■ LQRGD 
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23. How much do you agree or disagree with the following statements concerning your school 
district? 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


Don?t 

Know 


a. 


Actively recruits capable teachers 


□ 


□ 


□ 


□ 


□ 


b. 


Has an effective process for hiring teachers 


□ 


□ 


□ 


□ 


□ 


c. 


Gives permanent status only to competent teachers 


□ 


□ 


□ 


□ 


□ 


d. 


Supports me as a principal in my evaluation of 
teacher performance 


□ 


□ 


□ 


□ 


□ 


e. 


Works hard to improve the skills of ineffective 
teachers 


□ 


□ 


□ 


□ 


□ 


f. 


When appropriate, attempts to counsel ineffective 
teachers to leave the profession 


□ 


□ 


□ 


□ 


□ 


g- 


Has a teacher placement process that takes student 
needs into consideration 


□ 


□ 


□ 


□ 


□ 


h. 


Supports employee interest in additional education 
and certification 


□ 


□ 


□ 


□ 


□ 


i. 


Provides up-to-date instructional materials 


□ 


□ 


□ 


□ 


□ 


j- 


Provides enough instructional materials for all 
students 


□ 


□ 


□ 


□ 


□ 


k. 


Has a rigorous principal selection process 


□ 


□ 


□ 


□ 


□ 


1 . 


Provides AB 75 training to all principals 


□ 


□ 


□ 


□ 


□ 


m. 


Provides ongoing professional development to all 
principals 


□ 


□ 


□ 


□ 


□ 


n. 


Provides opportunities for principals to collaborate 
together 


□ 


□ 


□ 


□ 


□ 
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24. How much do you agree or disagree with the following statements about your school 
district? 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


Don?t 

Know 


a. 


Has a curriculum aligned with state standards 


□ 


□ 


□ 


□ 


□ 


b. 


Has a coherent grade-by -grade curriculum it uses 
for all schools 


□ 


□ 


□ 


□ 


□ 


c. 


Provides teacher professional development aligned 
with the district curriculum 


□ 


□ 


□ 


□ 


□ 


d. 


Has a clear expectation for student performance 
aligned with the curriculum 


□ 


□ 


□ 


□ 


□ 


e. 


Evaluates me as a principal based on the extent to 
which instruction in my school aligns with the 
curriculum 


□ 


□ 


□ 


□ 


□ 


f. 


Evaluates me as a principal based on student 
performance 


□ 


□ 


□ 


□ 


□ 


g- 


Has district staff highly skilled at curriculum and 
instruction 


□ 


□ 


□ 


□ 


□ 


h. 


Has district staff highly skilled at financial 
management 


□ 


□ 


□ 


□ 


□ 


i. 


Has district staff highly skilled at the analysis of 
performance data 


□ 


□ 


□ 


□ 


□ 


j- 


Expects that all schools in the district improve 
student achievement 


□ 


□ 


□ 


□ 


□ 


k. 


Has a superintendent and board that work together 
effectively 


□ 


□ 


□ 


□ 


□ 


1 . 


Maintains constructive relationships with employee 
unions 


□ 


□ 


□ 


□ 


□ 
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SECTION IV: Assessment and Data 

25. In what form do you receive CAT-6/CST data? (Please check all that apply.) 



a. Individually for all students 


□ 


b. A summary of all students across grade levels 


□ 


c. Disaggregated by specific skill/academic content 


□ 


d. Disaggregated by student subgroup 


□ 


e. I do not receive data for my students 


□ 



a. How do you use CAT-6/CST assessment data? (Please check all that apply.) 



a. To evaluate the progress of students 


□ 


b. To inform and communicate with parents 


□ 


c. To identify struggling students 


□ 


d. To develop strategies to move students from below basic and 
basic into proficiency 


□ 


e. To compare grades within the school 


□ 


f. To examine school-wide instructional issues 


□ 


g. To identify teachers who need instructional improvement 


□ 


h. None of the above 


□ 



b. Please indicate how your district uses CAT-6/CST assessment data. (Please check all that 
apply.) 



a. To set school-wide goals for student achievement 


□ 


b. To evaluate your curriculum programs 


□ 


c. To evaluate teachers? practices 


□ 


d. To compare grades and classrooms in your school 


□ 


e. To compare subgroups of students in your school 


□ 


f. To compare your school to similar schools 


□ 


g. To examine trends in your school?s performance 


□ 


h. None of the above 


□ 
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26. In what form do you receive CELDT assessment data? (Please check all that apply.) 



a. Individually for all students 


□ 


b. A summary of all students across grade levels 


□ 


c. Disaggregated by specific skill/academic content 


□ 


d. Disaggregated by student subgroup 


□ 


e. I do not receive data for my students 


□ 



a. How do you use CELDT assessment data? (Please check all that apply.) 



a. To evaluate the progress of students 


□ 


b. To inform and communicate with parents 


□ 


c. To identify struggling students 


□ 


d. To develop strategies to move students from below basic and 
basic in to proficiency 


□ 


e. To compare grades within the school 


□ 


f. To examine school-wide instructional issues 


□ 


g. To identify teachers who need instructional improvement 


□ 


h. None of the above 


□ 



b. Please indicate how your district uses CELDT assessment data. (Please check all that 
apply.) 



a. To set school-wide goals for student achievement 


□ 


b. To evaluate your curriculum programs 


□ 


c. To evaluate teachers? practices 


□ 


d. To compare grades and classrooms in your school 


□ 


e. To compare subgroups of students in your school 


□ 


f. To compare your school to similar schools 


□ 


g. To examine trends in your school?s performance 


□ 


h. None of the above 


□ 
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27. For which of the following curriculum programs, if any, does your school administer 
curriculum program assessments ? (Please check all that apply.) 



Language Arts 

Mathematics 

None of the above 



□ 

□ 

□ 



If None of the above, Go to Question 29 on the next page. 



a. Which, if any, of these curriculum program assessments are required by the district? 



Language Arts 

Mathematics 

None of the above 



□ 

□ 

□ 



b. In what form do you receive data from curriculum program assessments ? (Please check 
all that apply.) 



a. Individually for all students 


□ 


b. A summary of all students across grade levels 


□ 


c. Disaggregated by specific skill/academic content 


□ 


d. Disaggregated by student subgroup 


□ 


e. I do not receive data for my students 


□ 



c. How do you use curriculum program assessment data? (Please check all that apply.) 



a. To evaluate the progress of students 


□ 


b. To inform and communicate with parents 


□ 


c. To identify struggling students 


□ 


d. To develop strategies to move students from below basic and 
basic into proficiency 


□ 


e. To compare grades within the school 


□ 


f. To examine school-wide instructional issues 


□ 


g. To identity teachers who need instructional improvement 


□ 


h. None of the above 


□ 
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28. Please indicate how your district uses your school?s curriculum program assessment data. 

( Please check all that apply.) 



a. To set school-wide goals for student achievement 


□ 


b. To evaluate your curriculum programs 


□ 


c. To evaluate teachers? practices 


□ 


d. To compare grades and classrooms in your school 


□ 


e. To compare subgroups of students in your school 


□ 


f. To compare your school to similar schools 


□ 


g. To examine trends in your school?s performance 


□ 


h. None of the above 


□ 



29. Does your school administer assessments in Language Arts or Mathematics that were 
developed by the district ? 

Yes, for Language Arts only 

Yes, for Mathematics only 

Yes, for both Language Arts and Mathematics 
No 

a. Are these assessments required by the district ? 



.□ 

.□ 

.□ 

.□ If No, Go to Question 30 on page 19. 



Yes, for Language Arts only 

Yes, for Mathematics only 

Yes, for both Language Arts and Mathematics ... 
No 



□ 

□ 

□ 

□ 
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b. In what form do you receive data from these district developed assessments ? ( Please 
check all that apply.) 



a. Individually for all students 


□ 


b. A summary of all students across grade levels 


□ 


c. Disaggregated by specific skill/academic content 


□ 


d. Disaggregated by student subgroup 


□ 


e. I do not receive data for my students 


□ 



c. How do you use data from these district developed assessments ? (Please check all that 
apply.) 



a. To evaluate the progress of students 


□ 


b. To inform and communicate with parents 


□ 


c. To identify struggling students 


□ 


d. To develop strategies to move students from below basic and 
basic into proficiency 


□ 


e. To compare grades within the school 


□ 


f. To examine school-wide instructional issues 


□ 


g. To identity teachers who need instructional improvement 


□ 


h. None of the above 


□ 



d. Please indicate how your district uses these district developed assessment data about 
your school. (Please check all that apply.) 



a. To set school-wide goals for student achievement 


□ 


b. To evaluate your curriculum programs 


□ 


c. To evaluate teachers? practices 


□ 


d. To compare grades and classrooms in your school 


□ 


e. To compare subgroups of students in your school 


□ 


f. To compare your school to similar schools 


□ 


g. To examine trends in your school?s performance 


□ 


h. None of the above 


□ 
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30. Does your school administer other commercial assessments in Language Arts or 
Mathematics? 



Yes, for Language Arts only 

Yes, for Mathematics only 

Yes, for both Language Arts and Mathematics ... 
No 



□ 

□ 

□ 

□ 



If No, Go to Question 3 1 on page 2 1 . 



a. Are these other commercial assessments required by the district? 

Yes, for Language Arts only □ 

Yes, for Mathematics only □ 

Yes, for both Language Arts and Mathematics ... □ 

No 



b. In what form do you receive data from these other commercial assessments ? (Please 
check all that apply.) 



a. Individually for all students 


□ 


b. A summary of all students across grade levels 


□ 


c. Disaggregated by specific skill/academic content 


□ 


d. Disaggregated by student subgroup 


□ 


e. I do not receive data for my students 


□ 
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c. How do you use these other commercial assessment data? ( Please check all that apply.) 



a. To evaluate the progress of students 


□ 


b. To inform and communicate with parents 


□ 


c. To identify struggling students 


□ 


d. To develop strategies to move students from below basic and 
basic into proficiency 


□ 


e. To compare grades within the school 


□ 


f. To examine school-wide instructional issues 


□ 


g. To identity teachers who need instructional improvement 


□ 


h. None of the above 


□ 



d. Please indicate how your district uses these other commercial assessment data. ( Please 
check all that apply.) 



a. To set school-wide goals for student achievement 


□ 


b. To evaluate your curriculum programs 


□ 


c. To evaluate teachers? practices 


□ 


d. To compare grades and classrooms in your school 


□ 


e. To compare subgroups of students in your school 


□ 


f. To compare your school to similar schools 


□ 


g. To examine trends in your school?s performance 


□ 


h. None of the above 


□ 
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31. How frequently do you review assessment data (of any type)? 









Every 


Every 


A Few 












3-4 


6-8 


Times 


Once a 








Weekly 


Weeks 


Weeks 


a Year 


Year 


Never 


a. 


I review assessment data independently 


□ 


□ 


□ 


□ 


□ 


□ 


b. 


I review assessment data with teachers in 
their grade levels 


□ 


□ 


□ 


□ 


□ 


□ 


c. 


I review assessment data with teachers 
across grade levels 


□ 


□ 


□ 


□ 


□ 


□ 


d. 


I review assessment data with individual 
teachers 


□ 


□ 


□ 


□ 


□ 


□ 


32. 


How often do you do each of the following? 


















Every 


Every 


A Few 












3-4 


6-8 


Times 


Once a 








Weekly 


Weeks 


Weeks 


a Year 


Year 


Never 


a. 


Meet with individual teachers to review 
assessment data for individual students 


□ 


□ 


□ 


□ 


□ 


□ 


b. 


Use assessment data to set goals for 
individual student achievement 


□ 


□ 


□ 


□ 


□ 


□ 


c. 


Use assessment data to develop strategies 
to help selected students reach goals 


□ 


□ 


□ 


□ 


□ 


□ 


d. 


Use assessment data to follow up on 
progress of selected students 


□ 


□ 


□ 


□ 


□ 


□ 


e. 


Use assessment data to determine 
professional development teachers need 
to improve in a particular area 


□ 


□ 


□ 


□ 


□ 


□ 
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33. How much do you agree or disagree with each of the following statements about student 
subgroups? 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. Teachers use data to analyze student achievement 
by subgroup (e.g., racial/ethnic, EL) 


□ □ □ □ 


b. Teachers set measurable goals for student 
achievement by subgroup 


□ □ □ □ 


c. Teachers receive professional development that 
focuses on using assessment data by subgroup to 
improve student performance 


□ □ □ □ 


d. Our school sets measurable goals for student 
achievement by subgroup 


□ □ □ □ 


e. Our school dedicates time at staff meetings to 
discuss student achievement by subgroup 


□ □ □ □ 
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SECTION V: Professional Development 

34. How much have the following types of professional development sessions influenced your 
practices? ( N/A =Never Attended ) 







A 


A 


A 










Great 


Moderate 


Small 










Amount 


Amount 


Amount 


None 


N/A 


a. 


University course(s) related to your role as 
principal 


□ 


□ 


□ 


□ 


□ 


b. 


Visits to other schools designed to improve 
your work as principal 


□ 


□ 


□ 


□ 


□ 


c. 


Individual or collaborative research on a topic 
of interest to you 


□ 


□ 


□ 


□ 


□ 


d. 


Mentoring and/or peer observation and 
coaching of principals 


□ 


□ 


□ 


□ 


□ 


e. 


Participating in a principal network 


□ 


□ 


□ 


□ 


□ 


f. 


Workshops or conferences related to your role 
as principal 


□ 


□ 


□ 


□ 


□ 


g- 


Attending ACSA?s principal institute 


□ 


□ 


□ 


□ 


□ 


h. 


Completing AB 75 principal training 


□ 


□ 


□ 


□ 


□ 


i. 


District training/institutes 


□ 


□ 


□ 


□ 


□ 


j- 


Other (please specify): 


□ 


□ 


□ 


□ 


□ 



35. How much do you agree or disagree with each of the following statements? 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


Over the last 12 months, my professional development has: 


a. Been sustained and coherently focused 


□ 


□ 


□ 


□ 


b. Been closely connected to my school?s instructional goals 


□ 


□ 


□ 


□ 


c. Included enough time to discuss how to apply new ideas/ 
practices in my school 


□ 


□ 


□ 


□ 


d. Helped me better understand the needs of my teachers 


□ 


□ 


□ 


□ 


e. Helped me identify strategies to better meet the needs of 
struggling students 


□ 


□ 


□ 


□ 


f. Provided opportunities to work with principals in other 
schools 


□ 


□ 


□ 


□ 
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36. From the list below, please select your top three priorities for your own additional 
professional development. 



a. Developing a school plan or shared vision 


□ 


b. Promoting shared decision-making 


□ 


c. Involving and providing support to parents 


□ 


d. Fundraising/grant writing 


□ 


e. Implementing a specific instructional program 


□ 


f. Evaluating teachers? instruction 


□ 


g. Implementing a standards-based curriculum 


□ 


h. Addressing multicultural/diversity issues 


□ 


i. Using assessment data 


□ 


j . Training and instructional strategies for EL students 


□ 


k. Changes in state/federal accountability requirements 


□ 


1. Financial management 


□ 
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SECTION VI: About You 

The final set of questions will help us better describe who participated in the study. This information 
will be kept strictly confidential! 



37. Including this school year, how many years have you been: (If none, enter “00”.) 

a. A Principal Year(s) 

b. A Principal at this school Year(s) 

c. A Principal in this district 

d. An Elementary Assistant Principal 

e. A Middle/High School Principal 

f. A Middle/High School Assistant Principal 

g. A District Administrator 

h. A Full-time Teacher 

i. A Substitute Teacher 



Year(s) 

Year(s) 

Year(s) 

Year(s) 

Year(s) 

Year(s) 

Year(s) 



38. What grades have you taught? ( Please check all that apply.) 

N/A K 1 2 3 4 5 6 7 or higher □ 



39. What is the highest level of formal education you have completed? 



Bachelor?s degree 
Master?s degree.... 
Doctoral degree.... 



□ 

□ 
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40. Which of the following teaching credentials do you have? ( Please check all that apply.) 



a. Regular or standard certificate for California 


□ 


b. Regular or standard certificate for another state 


□ 


c. B-CLAD/ CLAD 


□ 


d. Hughes (SB 1969, 395) 


□ 


e. Education Specialist Instruction Credential 


□ 


f. National Board Certification 


□ 


g. Preliminary Credential 


□ 


h. Emergency Permit 


□ 


i. Administrative 


□ 


j. Other (please specify): 


□ 





41. What is your gender? 

Female □ 

Male 

42. What is your age? 

Years 

43. Are you comfortable conversing in a language other than English? 

Yes 

No □ — ►If No, Go to Question 45 on the next page. 
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44. In what other languages are you comfortable conversing? ( Please check all that apply.) 



a. Spanish 


□ 


b. Vietnamese 


□ 


c. Hmong 


□ 


d. Cantonese 


□ 


e. Filipino (Pilipino or Tag) 


□ 


f. Other (please specify ): 


□ 





45. To which of the following groups do you belong? (Please check all that apply.) 



a. American Indian or Alaska Native 


□ 


b. Asian 


□ 


c. Black or African American 


□ 


d. Filipino 


□ 


e. Latino/Hispanic 


□ 


f. Native Hawaiian or Other Pacific Islander 


□ 


g. White 


□ 


h. Other (please specify ): 


□ 
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46. In your opinion, what are the three most effective things your school has done to 
improve student achievement? 

Strategy #1: 



Strategy #2: 



Strategy #3: 
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INSTRUCTIONS 



□ When marking your responses on this survey, please use a number 2 pencil (preferable) or a 
ballpoint pen with dark ink. Do not use a felt tip pen or a Sharpie marker. 

□ Please indicate your answers by either checking {■/) or making an “X ” in the appropriate 
boxes. Write clearly and distinctly, avoiding any extraneous marks on the page. 

□ If you would like to change an answer, please erase any pencil marking completely or 
clearly strike through a mark made with a pen. 

□ For items that ask you to fill in a written response, please print your response in capital 
letters using one letter per box provided. 

When you have completed the survey, please put the survey in the enclosed envelope, seal it, 

and return it to the person whose name appears on the envelope. 

Thank you very much for taking the time to complete this survey! 
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SECTION I: School Context 

1. How much do you agree or disagree with each of the following statements? 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. 


Our school has a clear vision that is focused on student 
learning outcomes 


□ 


□ 


□ 


□ 


b. 


Our school has well defined plans for instructional 
improvement 


□ 


□ 


□ 


□ 


c. 


Our school assesses the effectiveness of our plans for 
instructional improvement 


□ 


□ 


□ 


□ 


2. 


How much do you agree or disagree with each of the following statements? 






Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. 


At our school, there is an enforced attendance policy 


□ 


□ 


□ 


□ 


b. 


At our school, there is an enforced dress code for 
students 


□ 


□ 


□ 


□ 


c. 


At our school, there are enforced rules for student 
behavior 


□ 


□ 


□ 


□ 


d. 


At our school, there is a zero tolerance policy toward 
bullying 


□ 


□ 


□ 


□ 


e. 


At our school, students respect cultural differences 


□ 


□ 


□ 


□ 


3. 


How much do you agree or disagree with each of the following statements? 






Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. 


My classroom instruction is guided by state standards 


□ 


□ 


□ 


□ 


b. 


Our school has identified essential/key standards 


□ 


□ 


□ 


□ 


c. 


Our school uses a standards-based report card 


□ 


□ 


□ 


□ 


d. 


Meeting our API growth target is a priority 


□ 


□ 


□ 


□ 


e. 


Meeting our API subgroup (e.g., racial/ethnic, EL) 
targets is a priority 


□ 


□ 


□ 


□ 


f. 


Meeting our AFP subgroup (e.g., racial/ethnic, EL) 
targets is a priority 


□ 


□ 


□ 


□ 


g- 


Our school sets measurable goals for EXCEEDING the 
mandated API subgroup growth targets 


□ 


□ 


□ 


□ 
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4. How frequently do you do each of the following with other teachers in your school ? 





Once or 
More 
a Week 


Once or 
Twice 
a Month 


A Few 
Times 
a Year 


Once 
a Year 


Never 


a. 


Share and discuss teaching methods 


□ 


□ 


□ 


□ 


□ 


b. 


Discuss particular lessons that were not very 
successful 


□ 


□ 


□ 


□ 


□ 


c. 


Examine the scope or sequence of curricular 
topics 


□ 


□ 


□ 


□ 


□ 


d. 


Review a grade-level pacing calendar 


□ 


□ 


□ 


□ 


□ 


e. 


Share and discuss student work 


□ 


□ 


□ 


□ 


□ 



5. How much do you agree or disagree with each of the following statements? 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. 


I have detailed knowledge of the content covered by 
other teachers at my school 


□ 


□ 


□ 


□ 


b. 


I have detailed knowledge of the instructional methods 
used by other teachers at my school 


□ 


□ 


□ 


□ 


c. 


There is consistency in curriculum and instruction a t the 
same grade level 


□ 


□ 


□ 


□ 


d. 


There is alignment in curriculum and instruction across 
different grade levels 


□ 


□ 


□ 


□ 


e. 


When we start a new instructional program, we follow up 
to see how it is impacting student achievement 


□ 


□ 


□ 


□ 



6. How much do you agree or disagree with each of the following statements? 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. 


Teachers take responsibility for student achievement 


□ 


□ 


□ 


□ 


b. 


Teachers are committed to improving student 
achievement 


□ 


□ 


□ 


□ 


c. 


Teachers communicate to students that education is 
important 


□ 


□ 


□ 


□ 


d. 


Teachers provide support to struggling teachers 


□ 


□ 


□ 


□ 


e. 


Teachers are involved in making important decisions at 
this school 


□ 


□ 


□ 


□ 


f. 


Teachers discuss assumptions about race and student 
achievement 


□ 


□ 


□ 


□ 



■ CVFWZ 



Page 2 



H Improving Achievement for Low-Income Students: What Makes a Difference? H 



7. How much influence do teachers at your school have in determining the following school 
policies? 





Strong 

Influence 


Moderate 

Influence 


Minor 

Influence 


No 

Influence 


a. 


Content of professional development 


□ 


□ 


□ 


□ 


b. 


Use of discretionary school funds 


□ 


□ 


□ 


□ 


c. 


Selection of curriculum and instructional programs 


□ 


□ 


□ 


□ 


d. 


Selection of supplemental books and other materials 


□ 


□ 


□ 


□ 


e. 


Hiring of new teachers 


□ 


□ 


□ 


□ 


f. 


Hiring of a new principal 


□ 


□ 


□ 


□ 



SECTION II: Role of the Principal and the District 



Principal Role 

8. How frequently does your principal do each of the following? 





Once or 
More 
a Week 


Once or 
Twice 
a 

Month 


A Few 
Times 
a Year 


Once 
a Year 


Never 


a. 


Participates in a grade-level meeting 


□ 


□ 


□ 


□ 


□ 


b. 


Formally evaluates my teaching 


□ 


□ 


□ 


□ 


□ 


c. 


Conducts a walkthrough of my classroom 


□ 


□ 


□ 


□ 


□ 


d. 


Teaches a demonstration/model class 


□ 


□ 


□ 


□ 


□ 
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9. How much do you agree or disagree with each of the following statements about your 
principal ? 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. 


Communicates a clear vision for our school 


□ 


□ 


□ 


□ 


b. 


Makes expectations clear to teachers for meeting 
academic achievement goals 


□ 


□ 


□ 


□ 


c. 


Is a knowledgeable source concerning standards and 
curriculum 


□ 


□ 


□ 


□ 


d. 


Sets high standards for student learning 


□ 


□ 


□ 


□ 


e. 


Provides support for classroom discipline and order 


□ 


□ 


□ 


□ 


f. 


Ensures that teachers have time for planning 


□ 


□ 


□ 


□ 


g- 


Praises and recognizes teachers 


□ 


□ 


□ 


□ 


h. 


Encourages teachers to take a leadership role in our 
school 


□ 


□ 


□ 


□ 


i. 


Provides teachers with adequate classroom materials 


□ 


□ 


□ 


□ 


j- 


Ensures that teachers receive adequate professional 
development to improve instruction 


□ 


□ 


□ 


□ 


k. 


Builds strong relationships with parents 


□ 


□ 


□ 


□ 
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District Role 

10. How much do you agree or disagree with each of the following statements about your 
district? 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


Don?t 

Know 


a. 


Supports our school in achieving its mission 


□ 


□ 


□ 


□ 


□ 


b. 


Prioritizes student learning and achievement 


□ 


□ 


□ 


□ 


□ 


c. 


Provides information about curriculum 
standards 


□ 


□ 


□ 


□ 


□ 


d. 


Provides student achievement data 


□ 


□ 


□ 


□ 


□ 


e. 


Implements instructional improvement 
strategies 


□ 


□ 


□ 


□ 


□ 


f. 


Addresses the instructional needs of English 
Language Learner students at our school 


□ 


□ 


□ 


□ 


□ 


g- 


Gives teachers opportunities to participate in 
decision-making at the district level (e.g., 
district level committees) 


□ 


□ 


□ 


□ 


□ 


h. 


Offers useful professional development 
activities 


□ 


□ 


□ 


□ 


□ 


i. 


Encourages teachers to take a leadership role 
in the district 


□ 


□ 


□ 


□ 


□ 


j- 


Praises and recognizes teachers 


□ 


□ 


□ 


□ 


□ 


k. 


Is aware of the challenges and problems our 
school faces 


□ 


□ 


□ 


□ 


□ 


1 . 


Builds the community?s confidence in our 
school 


□ 


□ 


□ 


□ 


□ 


m. 


Ensures that district staff visits our school at 
least once a year 


□ 


□ 


□ 


□ 


□ 
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SECTION III: Core Instruction 

11. In a typical day, how much instructional time do you spend on Language Arts ? 



1 hour or less 


□ 


1.5 hours 


□ 


2 hours 


□ 


2.5 hours 


□ 


3 hours or more 


□ 



b. In a typical day, how much instructional time do you spend on Mathematics ? 



30 minutes or less 


□ 


45 minutes 


□ 


60 minutes 


□ 


75 minutes 


□ 


90 minutes or more 


□ 



12. How frequently do you use the following Language Arts curriculum programs in your 
classroom? 





Daily 


Once or 
Twice 
a Week 


Once or 
Twice 
a 

Month 


A Few 
Times 
a Year 


Never 


a. 


Harcourt Brace Spelling 


□ 


□ 


□ 


□ 


□ 


b. 


Houghton Mifflin 


□ 


□ 


□ 


□ 


□ 


c. 


McGraw Hill/MacMillan?s Reading 


□ 


□ 


□ 


□ 


□ 


d. 


Open Court 


□ 


□ 


□ 


□ 


□ 


e. 


Phonics SRA 


□ 


□ 


□ 


□ 


□ 


f. 


Scholastic Phonics 


□ 


□ 


□ 


□ 


□ 


g- 


Scott Foresman 


□ 


□ 


□ 


□ 


□ 


h. 


Write Source Language Program 


□ 


□ 


□ 


□ 


□ 


i. 


Other (please specify ): 

mnxinn:i::riTii 


□ 


□ 


□ 


□ 


□ 
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13. How frequently do you use the following Math curriculum programs in your classroom? 





Daily 


Once or 
Twice 
a Week 


Once or 
Twice 
a 

Month 


A Few 
Times 
a Year 


Never 


a. 


Harcourt Brace 


□ 


□ 


□ 


□ 


□ 


b. 


Houghton Mifflin 


□ 


□ 


□ 


□ 


□ 


c. 


McDougal Little 


□ 


□ 


□ 


□ 


□ 


d. 


McGraw Hill 


□ 


□ 


□ 


□ 


□ 


e. 


Prentice Hall 


□ 


□ 


□ 


□ 


□ 


f. 


Progress in Math 


□ 


□ 


□ 


□ 


□ 


g- 


Saxon Math 


□ 


□ 


□ 


□ 


□ 


h. 


Scott Foresman 


□ 


□ 


□ 


□ 


□ 


i. 


Success with MathCoach 


□ 


□ 


□ 


□ 


□ 


j- 


Other (please specify ): 














1 | | ] || | | | | | | | 


□ 


□ 


□ 


□ 


□ 
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14. How much do you agree or disagree with each of the following statements? 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. This subject is protected from unnecessary interruptions 


Language Arts 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 


b. This subject is integrated with other subjects 


Language Arts 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 


c. The curriculum materials meet the needs of the majority of my students 


Language Arts 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 


d. The curriculum materials are aligned with state standards 


Language Arts 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 


e. I have opportunities to experiment with instructional strategies 


Language Arts 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 


f. I have opportunities to integrate culturally relevant 


materials 








Language Arts 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 


g. I have opportunities to incorporate creative elements into my teaching 


Language Arts 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 


h. I have received adequate training in the current curriculum program 


Language Arts 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 


i. My classroom has the current version of the textbook 


Language Arts 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 


j. Every student in my classroom has a copy of the textbook 


Language Arts 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 
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15. How frequently do you do each of the following? 





Daily 


Once or 
Twice 
a Week 


Once or 
Twice 
a 

Month 


A Few 
Times 
a Year 


Never 


a. Follow a lesson plan provided by the curricu 


um program, making few or no adjustments 


Language Arts 


□ 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 


□ 


b. Modify a lesson plan provided by the curriculum program to better fit the needs of my students 


Language Arts 


□ 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 


□ 


c. Use texts provided by the curriculum program 


Language Arts 


□ 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 


□ 


d. Follow a pacing plan provided for my grade 


level 


Language Arts 


□ 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 


□ 


e. Map state curriculum standards onto my lesson plans 


Language Arts 


□ 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 


□ 
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16. How frequently do you administer the following assessments? 





Weekly 


Every 3-4 
Weeks 


Every 6-8 
Weeks 


A Few 
Times 
a Year 


Never 


a. Assessments created by myself or others in my school 


Language Arts 


□ 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 


□ 


b. Assessments from the curriculum program 


Language Arts 


□ 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 


□ 


c. Assessments developed by our district 


Language Arts 


□ 


□ 


□ 


□ 


□ 


Mathematics 


□ 


□ 


□ 


□ 


□ 


d. Other commercial assessments 


Language Arts (please specify ): 












1 | || | || M 1 1 1 


□ 


□ 


□ 


□ 


□ 


Mathematics ( please specify): 












1 1 1 1 1 1 1 1 1 1 1 


□ 


□ 


□ 


□ 


□ 



SECTION IV: Instructional Strategies 
17. How frequently does each of the following occur? 





Once or 
More 
a Week 


Once or 
Twice 
a Month 


A Few 
Times 
a Year 


Once 
a Year 


Never 


a. 


I inform parents about my curriculum and 
instruction 


□ 


□ 


□ 


□ 


□ 


b. 


I provide suggestions for activities that 
parents can do at home with their child 


□ 


□ 


□ 


□ 


□ 


c. 


I create homework assignments that involve 
parents 


□ 


□ 


□ 


□ 


□ 


d. 


I have parents who provide instructional 
support in my classroom 


□ 


□ 


□ 


□ 


□ 
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18. How frequently do you do the following to address the needs of students who are struggling 
academically ? 





Once or 
More 
a Week 


Once or 
Twice 
a Month 


A Few 
Times 
a Year 


Once 
a Year 


Never 


a. 


Adjust my pace in the unit for the entire class 
to address the needs of struggling students 


□ 


□ 


□ 


□ 


□ 


b. 


Review key concepts in the unit for the entire 
class to address the needs of struggling 
students 


□ 


□ 


□ 


□ 


□ 


c. 


Use same-level achievement grouping to 
address the needs of struggling students 


□ 


□ 


□ 


□ 


□ 


d. 


Use mixed achievement grouping to address 
the needs of struggling students 


□ 


□ 


□ 


□ 


□ 


e. 


Use individualized instruction during class to 
address the needs of struggling students 


□ 


□ 


□ 


□ 


□ 


f. 


Provide individual assistance outside of class 
to address the needs of struggling students 


□ 


□ 


□ 


□ 


□ 



19. How frequently do you refer struggling students in your classroom to the following kinds of 
services? 







Once or 
More 
a Week 


Once or 
Twice 
a Month 


A Few 
Times 
a Year 


Once 
a Year 


Never 


a. 


Diagnostic assessment 


□ 


□ 


□ 


□ 


□ 


b. 


Individual tutoring from a specialist 


□ 


□ 


□ 


□ 


□ 


c. 


Individual tutoring from an aide 


□ 


□ 


□ 


□ 


□ 


d. 


Individual tutoring from a volunteer 


□ 


□ 


□ 


□ 


□ 


e. 


Pull-out instruction in a small group 


□ 


□ 


□ 


□ 


□ 


f. 


Additional instruction in a before-school, 
after-school, or Saturday program 


□ 


□ 


□ 


□ 


□ 


g- 


Additional instruction in a summer school/ 
intersession program 


□ 


□ 


□ 


□ 


□ 
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20. In a typical week, how much additional instruction in Language Arts, on average, do 
struggling students receive outside of class ? 



None 


□ 


30 minutes 


□ 


60 minutes 


□ 


90 minutes 


□ 


120 minutes or more 


□ 



b. In a typical week, how much additional instruction in Mathematics , on average, do 
struggling students receive outside of class ? 



None 


□ 


30 minutes 


□ 


60 minutes 


□ 


90 minutes 


□ 


120 minutes or more 


□ 



SECTION V: English Learners 

21. Do you currently have any English Learner (EL) students in your class? 

Yes 

No □ — ►If No, Go to Question 33 on page 16. 



22. Among your EL students, how many of your students are in the following CELDT levels? 

(If none, enter “00”.) 



a. 

b. 

c. 

d. 



Levels 1-2 (Beginning) 
Level 3 (Intermediate) .. 
Levels 4-5 (Advanced) . 
Don?t Know 



Student(s) 

Student(s) 

Student(s) 

□ 
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23. Does your school provide explicit instruction in English Language Development to EL 
students? 

Yes 

No □ — ►If No, Go to Question 27 on the next page. 

24. How is explicit English Language Development (ELD) instruction delivered to your EL 
students? (Please check all that apply.) 



a. Whole class (ELs only) 


□ 


b. By ELD level within your class 


□ 


c. By ELD level through teaming 


□ 


d. Pull-out program (e.g., resource teacher) 


□ 



25. How much daily instructional time is specifically allotted for explicit English Language 
Development? 



None 


□ 


15 minutes 


□ 


30 minutes 


□ 


45 minutes 


□ 


60 minutes 


□ 


75 minutes or more 


□ 



26. Who teaches your EL students their explicit English Language Development instruction? 

(Please check all that apply.) 



a. Myself 


□ 


b. Resource Teacher 


□ 


c. Another Teacher with EL credential (CLAD, BCLAD, or similar) 


□ 


d. Another Teacher without EL credential 


□ 


e. Instructional aide 


□ 


f. Other (please specify): 


□ 







m NPXUR 



Page 13 



H Improving Achievement for Low-Income Students: What Makes a Difference? H 



27. In addition to their explicit English Language Development, how much supplemental 
instruction do EL students receive to move them to the next English proficiency level? 



None 


□ 


30 minutes 


□ 


60 minutes 


□ 


90 minutes 


□ 


120 minutes or more 


□ 



28. How are your EL students taught mathematics? (Please check all that apply.) 



a. In their native language 


□ 


b. In a mainstream classroom 


□ 


c. Using ESL or immersion techniques (SDAIE) 


□ 


d. With primary language assistance from an instructional aide 


□ 


e. With primary language assistance from the teacher 


□ 


f. With primary language assistance from a resource teacher 


□ 


g. Using mathematics material designed for EL students 


□ 


h. Using primary language math materials 


□ 



29. In addition to the CELDT, what types of assessment do you use for EL students? (Please 
check all that apply.) 



a. School level ELD diagnostic assessment 


□ 


b. Program-specific ELD diagnostic assessment 


□ 


c. SABE (Spanish Assessment of Basic Education) 


□ 


d. School level academic assessments in students? primary language 


□ 


e. School level academic assessment using techniques designed for 
EL students 


□ 


f. Other (please specify): 


□ 
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30. How frequently do you use the following EL curriculum programs in your classroom? 





Daily 


Once or 
Twice 
a Week 


Once or 
Twice 
a 

Month 


A Few 
Times 
a Year 


Never 


a. 


Fast Track Reading Program 


□ 


□ 


□ 


□ 


□ 


b. 


Glencoe/McGraw Hill 


□ 


□ 


□ 


□ 


□ 


c. 


Hampton Brown 


□ 


□ 


□ 


□ 


□ 


d. 


Holt Literature and Language Arts 


□ 


□ 


□ 


□ 


□ 


e. 


Houghton Mifflin 


□ 


□ 


□ 


□ 


□ 


f. 


Lectura 


□ 


□ 


□ 


□ 


□ 


g- 


McDougal Littell 


□ 


□ 


□ 


□ 


□ 


h. 


Prentice Hall 


□ 


□ 


□ 


□ 


□ 


i. 


Scholastic Read 


□ 


□ 


□ 


□ 


□ 


j- 


Scott Foresman 


□ 


□ 


□ 


□ 


□ 


k. 


SRA/ McGraw Hill 


□ 


□ 


□ 


□ 


□ 


1 . 


Other (please specify): 


□ 


□ 


□ 


□ 


□ 


31. 


How frequently do you meet with teachers to do the following? 








Once or 
More 
a Week 


Once or 
Twice 
a Month 


A Few 
Times 
a Year 


Once 
a Year 


Never 


a. 


Discuss the academic needs of your EL 
students 


□ 


□ 


□ 


□ 


□ 


b. 


Determine the best instructional approaches for 
your EL students 


□ 


□ 


□ 


□ 


□ 


c. 


Consult with an EL specialist 


□ 


□ 


□ 


□ 


□ 
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32. How have the following types of instructional assistance in EL influenced your teaching 
practices? (N/A=Not applicable; did not receive) 





A Great 
Amount 


A 

Moderate 

Amount 


A Small 
Amount 


N/A 


a. 


Assistance in EL instructional strategies from EL coaches 


□ 


□ 


□ 


□ 


b. 


Teachers who are succeeding with EL students present 
successful strategies 


□ 


□ 


□ 


□ 


c. 


Opportunities to observe other teachers 


□ 


□ 


□ 


□ 


d. 


Observation and feedback on your teaching of EL students 


□ 


□ 


□ 


□ 



SECTION VI: Kindergarten 

33. Do you currently have any kindergarten students in your class? 

Yes 

No □ — ►If No, Go to Question 35 on the next page. 

34. How much do you agree or disagree with each of the following statements? 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. 


I use play as an instructional activity 


□ 


□ 


□ 


□ 


b. 


Teaching social skills is a primary part of my curriculum 


□ 


□ 


□ 


□ 


c. 


Teaching academic content is a primary part of my 
curriculum 


□ 


□ 


□ 


□ 


d. 


The majority of my students have attended at least one 
year of preschool 


□ 


□ 


□ 


□ 


e. 


The majority of my students enter school academically 
prepared 


□ 


□ 


□ 


□ 


f. 


I spend the majority of class time teaching academic skills 


□ 


□ 


□ 


□ 


g- 


I use the state standards to guide what I teach 


□ 


□ 


□ 


□ 


h. 


The kindergarten curriculum standards are 
developmentally appropriate 


□ 


□ 


□ 


□ 


i. 


I spend time preparing my students for assessments 


□ 


□ 


□ 


□ 


j- 


I use a report card that includes student?s academic 
progress 


□ 


□ 


□ 


□ 
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SECTION VII: Assessment and Data 
35. Do your students take the CAT-6/CST ? 

Yes 

No □ — ►If No, Go to Question 36 on the next page. 

a. In what form do you receive CAT-6/CST assessment data? (Please check all that apply.) 



a. Individually for all students in my classroom 


□ 


b. A summary of all students across my grade level 


□ 


c. Disaggregated by specific skill/academic content for all 
students in my classroom 


□ 


d. Disaggregated by student subgroup for all students in 
my classroom 


□ 


e. I do not receive data for my students 


□ 



b. How do you use CAT-6/CST assessment data? (Please check all that apply.) 



a. To evaluate the progress of students 


□ 


b. To adjust my curriculum in areas where students encountered 
problems 


□ 


c. To inform parents of student progress 


□ 


d. To place students into instructional groups by achievement 
level 


□ 


e. To identify struggling students 


□ 


f. To develop strategies to move students from below basic and 
basic into proficiency 


□ 


g. None of the above 


□ 
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36. Do your students take the CELDT ? 

Yes 

No □ — ►If No, Go to Question 37 on the next page. 

a. In what form do you receive CELDT assessment data? (Please check all that apply.) 



a. Individually for all students in my classroom 


□ 


b. A summary of all students across my grade level 


□ 


c. Disaggregated by specific skill/academic content for all 
students in my classroom 


□ 


d. Disaggregated by student subgroup for all students in 
my classroom 


□ 


e. I do not receive data for my students 


□ 



b. How do you use CELDT assessment data? (Please check all that apply.) 



a. To evaluate the progress of students 


□ 


b. To adjust my curriculum in areas where students encountered 
problems 


□ 


c. To inform parents of student progress 


□ 


d. To place students into instructional groups by achievement 
level 


□ 


e. To identify struggling students 


□ 


f. To develop strategies to move students from below basic and 
basic into proficiency 


□ 


g. None of the above 


□ 
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37. Do your students take curriculum program assessments ? 

Yes 

No □ — ►If No, Go to Question 38 on the next page. 

a. In what form do you receive curriculum program assessment data? (Please check all that 
apply.) 



a. Individually for all students in my classroom 


□ 


b. A summary of all students across my grade level 


□ 


c. Disaggregated by specific skill/academic content for all 
students in my classroom 


□ 


d. Disaggregated by student subgroup for all students in 
my classroom 


□ 


e. I do not receive data for my students 


□ 



b. How do you use curriculum program assessment data? ( Please check all that apply.) 



a. To evaluate the progress of students 


□ 


b. To adjust my curriculum in areas where students encountered 
problems 


□ 


c. To inform parents of student progress 


□ 


d. To place students into instructional groups by achievement 
level 


□ 


e. To identify struggling students 


□ 


f. To develop strategies to move students from below basic and 
basic into proficiency 


□ 


g. None of the above 


□ 



■ TJEPO 



Page 19 




H Improving Achievement for Low-Income Students: What Makes a Difference? H 



38. Do your students take district developed assessments ? 

Yes 

No □ — ►If No, Go to Question 39 on the next page. 

a. In what form do you receive data from district developed assessments ? ( Please check all 
that apply.) 



a. Individually for all students in my classroom 


□ 


b. A summary of all students across my grade level 


□ 


c. Disaggregated by specific skill/academic content for all 
students in my classroom 


□ 


d. Disaggregated by student subgroup for all students in 
my classroom 


□ 


e. I do not receive data for my students 


□ 



b. How do you use data from district developed assessments ? (Please check all that apply.) 



a. To evaluate the progress of students 


□ 


b. To adjust my curriculum in areas where students encountered 
problems 


□ 


c. To inform parents of student progress 


□ 


d. To place students into instructional groups by achievement 
level 


□ 


e. To identify struggling students 


□ 


f. To develop strategies to move students from below basic and 
basic into proficiency 


□ 


g. None of the above 


□ 



m UXHNC 
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39. Do your students take other commerical assessments ? 

Yes 

No □ — ►If No, Go to Question 40 on the next page. 

a. In what form do you receive data from other commercial assessments ? (Please check all 
that apply.) 



a. Individually for all students in my classroom 


□ 


b. A summary of all students across my grade level 


□ 


c. Disaggregated by specific skill/academic content for all 
students in my classroom 


□ 


d. Disaggregated by student subgroup for all students in 
my classroom 


□ 


e. I do not receive data for my students 


□ 



b. How do you use data from other commercial assessments ? (Please check all that apply.) 



a. To evaluate the progress of students 


□ 


b. To adjust my curriculum in areas where students encountered 
problems 


□ 


c. To inform parents of student progress 


□ 


d. To place students into instructional groups by achievement 
level 


□ 


e. To identify struggling students 


□ 


f. To develop strategies to move students from below basic and 
basic into proficiency 


□ 


g. None of the above 


□ 
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40. How frequently do you review assessment data (of any type)? 









Every 


Every 


A Few 












3-4 


6-8 


Times 


Once a 








Weekly 


Weeks 


Weeks 


a Year 


Year 


Never 


a. 


I review assessment data independently 


□ 


□ 


□ 


□ 


□ 


□ 


b. 


I review assessment data with teachers in 
my grade level 


□ 


□ 


□ 


□ 


□ 


□ 


c. 


I review assessment data with teachers 
across grade levels 


□ 


□ 


□ 


□ 


□ 


□ 


d. 


I review assessment data with my 
principal 


□ 


□ 


□ 


□ 


□ 


□ 


41. 


How often do you do each of the following? 


















Every 


Every 


A Few 












3-4 


6-8 


Times 


Once a 








Weekly 


Weeks 


Weeks 


a Year 


Year 


Never 


a. 


Meet with principal to review assessment 
data for individual students 


□ 


□ 


□ 


□ 


□ 


□ 


b. 


Use assessment data to set goals for 
individual student achievement 


□ 


□ 


□ 


□ 


□ 


□ 


c. 


Use assessment data to develop strategies 
to help selected students reach goals 


□ 


□ 


□ 


□ 


□ 


□ 


d. 


Use assessment data to follow up on 
progress of selected students 


□ 


□ 


□ 


□ 


□ 


□ 


e. 


Use assessment data to determine what 
professional development I need to 
improve in a particular area 


□ 


□ 


□ 


□ 


□ 


□ 



42. How much do you agree or disagree with each of the following statements about student 
subgroups ? 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. 


I use assessment data to analyze student 
achievement by subgroups (e.g., racial/ethnic, EL) 


□ 


□ 


□ 


□ 


b. 


I set measurable goals for student achievement by 
subgroup 


□ 


□ 


□ 


□ 


c. 


I receive professional development that focuses on 
using assessment data by subgroup to improve 
student achievement 


□ 


□ 


□ 


□ 


d. 


Our school sets measurable goals for student 
achievement by subgroup 


□ 


□ 


□ 


□ 


e. 


Our school dedicates time at staff meetings to 
discuss student achievement data by subgroup 


□ 


□ 


□ 


□ 



■ WKDMA 
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SECTION VIII: Professional Development 

43. In the last 12 months, how frequently did the following professional development activities 
occur? 







4 or 
More 


2 to 3 




Vi Day 








Days 


Days 


1 Day 


or Less 


No Days 


a. 


I watched an instructional leader (e.g., coach, 
coordinator) model instruction 


□ 


□ 


□ 


□ 


□ 


b. 


An instructional leader studied students? work 
and suggested ways to improve their learning of 
the subject matter 


□ 


□ 


□ 


□ 


□ 


c. 


An instructional leader demonstrated ways to 
assess student learning 


□ 


□ 


□ 


□ 


□ 


d. 


An instructional leader demonstrated use of 
student achievement data for improving 
instruction 


□ 


□ 


□ 


□ 


□ 


e. 


Another teacher observed me teach and gave 
me feedback 


□ 


□ 


□ 


□ 


□ 


f. 


I observed another teacher 


□ 


□ 


□ 


□ 


□ 



44. How have the following professional development sessions influenced your teaching 
practices? (N/A=Not applicable; did not receive professional development in this area) 





A 

Great 

Amount 


A 

Moderate 

Amount 


A 

Small 

Amount 


No 

Amount 


N/A 


a. 


Curriculum standards (AB 466) 


□ 


□ 


□ 


□ 


□ 


b. 


Multicultural or diversity issues 


□ 


□ 


□ 


□ 


□ 


c. 


Classroom management and student discipline 


□ 


□ 


□ 


□ 


□ 


d. 


School improvement planning 


□ 


□ 


□ 


□ 


□ 


e. 


Language Arts curriculum program 


□ 


□ 


□ 


□ 


□ 


f. 


Mathematics curriculum program 


□ 


□ 


□ 


□ 


□ 


g- 


English Language Development curriculum 
program 


□ 


□ 


□ 


□ 


□ 


h. 


Instructional strategies for English Learners 


□ 


□ 


□ 


□ 


□ 


i. 


Instructional strategies for multiple learning 
styles 


□ 


□ 


□ 


□ 


□ 


j- 


Understanding and using data from assessments 
to inform instruction 


□ 


□ 


□ 


□ 


□ 



M XOYFL 
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45. How much do you agree or disagree with each of the following statements? 





Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


Over the last 12 months , my professional development has: 


a. 


Been sustained and coherently focused 


□ 


□ 


□ 


□ 


b. 


Been closely connected to my school?s instructional goals 


□ 


□ 


□ 


□ 


c. 


Included enough time to discuss how to apply new ideas/ 
practices in my school 


□ 


□ 


□ 


□ 


d. 


Helped me better understand the needs of my students 


□ 


□ 


□ 


□ 


e. 


Helped me identify strategies to better meet the needs of 
my struggling students 


□ 


□ 


□ 


□ 


f. 


Included opportunities to work with teachers in my school 


□ 


□ 


□ 


□ 


g- 


Included opportunities to work with teachers in other 
schools 


□ 


□ 


□ 


□ 



46. From the list below, please select your top three priorities for your own additional 
professional development. 



a. Curriculum standards (AB 466) 


□ 


b. Multicultural or diversity issues 


□ 


c. Classroom management and student discipline 


□ 


d. School improvement planning 


□ 


e. Language Arts curriculum program 


□ 


f. Mathematics curriculum program 


□ 


g. English Language Development curriculum program 


□ 


h. Instructional strategies for English Learners 


□ 


i. Instructional strategies for multiple learning styles 


□ 


j . Understanding and using data from assessments to inform 
instruction 


□ 



■ YTOCB 
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SECTION IX: About You 

The final set of questions will help us better describe who participated in the study. This information 
will be kept strictly confidential! 



47. If this is your first year at this school, in what month did you begin teaching at this school? 

September 
October .... 

November 
December. 

January 

N/A 

48. Are you a long-term substitute teacher at this school? 

Yes 
No 

49. Are you employed on a shared contract? A shared contract is an arrangement through 
which two teachers each work part-time and share responsibility for a classroom. 

Yes 
No 



.□ 

.□ 

.□ 

.□ 

.□ 



50. Including this school year, how many years have you been teaching? (If none, please enter 
“ 00 ”.) 



a. 

b. 

c. 

d. 



Full-time 






Year(s) 

Year(s) 

Year(s) 

Year(s) 


As a substitute 






At your current school full-time 






At your current school as a substitute 









■ ZLCBM 
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51. At what grade level(s) are you currently teaching? ( Please check all that apply.) 



K 1 2 3 4 5 6 



52. How many years have you taught this current grade level? 

Years 

53. What were your previous teaching responsibilities? (Please check all that apply.) 

Grade Levels: K 1 □ 2 3 4 5 6 



54. Do you have a special program assignment in any of these areas? (Please check all that 
apply.) 



Special Education 


□ 


Bilingual 


□ 


Other (please specify): 


□ 





55. How many students are in your class? 

Students 



56. What is the highest level of formal education you have completed? 



Bachelor?s degree 
Master?s degree.... 
Doctoral degree.... 



□ 

□ 

□ 



■ AAFUL 



Page 26 




H Improving Achievement for Low-Income Students: What Makes a Difference? H 



57. Which of the following teaching credentials do you have? ( Please check all that apply.) 



a. Regular or standard certificate for California 


□ 


b. Regular or standard certificate for another state 


□ 


c. B-CLAD/ CLAD 


□ 


d. Hughes (SB 1969, 395) 


□ 


e. Education Specialist Instruction Credential 


□ 


f. National Board Certification 


□ 


g. Preliminary Credential 


□ 


h. Emergency Permit 


□ 


i. Administrative 


□ 


j. Other (please specify): 


□ 





58. What is your gender? 

Female □ 

Male 

59. What is your age? 

Years 

60. Are you comfortable conversing in a language other than English? 

Yes 

No □ — ►If No, Go to Question 62 on the next page. 



■ BBHSG 
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61. In what other languages are you comfortable conversing? ( Please check all that apply.) 



a. Spanish 


□ 


b. Vietnamese 


□ 


c. Hmong 


□ 


d. Cantonese 


□ 


e. Filipino (Pilipino or Tag) 


□ 


f. Other ( please specify): 


□ 





62. To which of the following groups do you belong? (Please check all that apply.) 



a. American Indian or Alaska Native 


□ 


b. Asian 


□ 


c. Black or African American 


□ 


d. Filipino 


□ 


e. Latino/Hispanic 


□ 


f. Native Hawaiian or Other Pacific Islander 


□ 


g. White 


□ 


h. Other (please specify): 


□ 





■ CCUTA 
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63. In your opinion, what are the three most effective things your school has done to 
improve student achievement? 

Strategy#!: 



Strategy #2: 



Strategy #3: 



Copyright 2005 EdSource, Inc. All rights reserved, www.edsource.org. 
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APPENDIX A 



Recruiting Schools and Districts for Study Participation 

EdSource took on the task of recruiting schools for the study because of its 28-year history in the state, 
and strong reputation and relationship with districts. The goal was to recruit a large number of schools 
within the 25 th to 35 th percentile band to participate, which would be challenging because of the time 
required by the school: the research team was asking that the principal plus 80% of the school’s K-5 
classroom teachers complete a long survey that would take approximately an hour of time. 

In October of 2004 EdSource contacted all district superintendents by mail and then by phone both to 
explain the study and obtain permission to contact the district’s elementary schools identified for our 
sample. While senior staff at EdSource made calls to key districts, they also recruited and hired local 
school board members who were familiar with EdSource to serve on a phone bank for the study. These 
informed individuals were effective because they understood the dynamics of the district and school 
environment and the importance of respecting it as they interacted with staff. School board callers served 
as our research liaisons and worked in shifts so that someone was available during most working hours if 
a district called for information. 

Once a district had given permission for EdSource to contact the targeted elementary school(s) about the 
study, EdSource mailed letters directly to the elementary schools, letting them know that their 
superintendent had approved the contact, explaining the importance and scope of the study, and asking 
them to participate. The letter was clear about the expectation for an 80% K-5 classroom teacher 
participation rate, and suggested that the school set aside an hour of staff meeting time in January, when 
the surveys were to be mailed, for survey administration. We suggested that a teacher or administrative 
support person at the school be designated as the “study coordinator” to issue and collect the teacher 
surveys and ensure confidentiality. The principal would receive and return his/her survey separately. 
EdSource offered incentives for participating: a $25 gift certificate to a bookstore for the study 
coordinator; and if 80% of a school’s classroom teachers and the principal returned completed surveys the 
school would receive $100 to use for any purpose, and the school would receive a copy of the lay report 
of the study findings when it was published. 

After receiving a letter requesting their school’s participation, some principals returned the enclosed fax- 
back form indicating that their school would join the study. But the majority of principals did not, 
necessitating one or more phone calls from the research liaisons. In either case, principals agreeing to 
participate were asked to send the name of a volunteer coordinator and a list of names of their K-5 
classroom teachers. This list let the researchers know how many surveys to send and served as a checklist 
for the school coordinator who would be retrieving the completed surveys. 

154 districts and 269 schools were recruited through this process. 



Survey Administration and Retrieval 

American Institutes for Research (AIR) researchers joined the research team in December of 2004 and 
helped oversee the survey retrieval process— partnering with EdSource to train the callers and to develop 
a script for them to use. From January through February, AIR mailed packets of surveys to the volunteer 
coordinators as each school made a commitment to participate. 

To assure teachers that their responses would remain confidential, we included blank envelopes so they 
could seal their completed surveys inside before returning them to the coordinator. Once coordinators 
had retrieved the teacher surveys, they would overnight them in one large envelope. The principal was 
given an envelope as well and encouraged to mail this back separately to avoid concern that the staff 
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member serving as coordinator might see the responses. Completed surveys were returned to AIR and 
then logged into the computer. Each schools’ surveys were identified with a similar code so AIR could 
aggregate one site’s results. If schools reported losing or misplacing surveys, EdSource or AIR sent 
additional ones coded with this same identifier. 

EdSource’ s bank of school board phone callers, or research liaisons, were a critical component both in 
getting schools to participate and in achieving such a teacher survey return rate. They followed up with 
phone calls to schools to ensure that as many surveys as possible were being returned. If schools needed 
help, they had access to an 800 number set up by AIR that routed them to the liaisons who fielded 
questions and gave encouragement and advice on how to get more teachers to return the surveys. Callers 
tracked information about schools on a computer log sheet so that if one caller’s shift ended, the caller 
serving in the next shift would have all the pertinent and updated information needed. Each caller 
recorded a detailed narrative of every conversation with the school so the next caller was familiar with 
that school’s situation before calling. AIR also developed tracking sheets in Excel that included an actual 
roster with teachers’ names and the number of teacher responses needed to achieve the 80% goal. AIR 
set up a networked computer system that enabled each caller to log into an individual remote desktop and 
access all the files as well as individual email accounts. This enabled callers to access the schools’ files 
quickly and communicate efficiently with other callers. The full team of callers plus AIR and EdSource 
personnel met once a week to share concerns and brainstorm solutions or alternative approaches to 
increase school participation and compliance 

Of the 154 districts and 269 schools recruited to particpate in the study, 145 districts and 257 schools 
returned a principal survey and a sufficient number of teacher surveys. The large majority of schools had 
a minimum of 80% of K-5 classroom teachers returning surveys; many schools returned 100%. 
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APPENDIX B 



Research Methodology and Analyses 

Researchers from each of the four collaborating organizations have met biweekly since March 2005 to 
discuss the data analyses, while the technical team has met more frequently to specify details of the 
analyses. Data file construction was carried out primarily by AIR under the direction of Ed Haertel, the 
study’s senior technical consultant. 

Using a weighted analysis assured that results were statistically representative of all public non-charter 
elementary schools in the 25 th to 35 th 2003-04 SCI percentile band. School demographic characteristics 
and test performance data (API and CSTs) were downloaded from databases available at 
http://api.cde.ca.gov/datafiles.asp and/or http://ayp.cde.ca.gov/datafiles.asp . Because the sampling design 
included stratification according to whether the district was an EdSource Information Service subscriber, 
this factored into the construction of the weighting variable. 

After data cleaning, teacher survey items were screened to assure that within-school consistency of 
responses was sufficient to provide useful measurements of school characteristics. Using the teacher- 
level data file, intra-class correlations for teachers within schools and corresponding reliabilities were 
calculated for each item, and fifteen teacher survey items (roughly 5 percent of all items on the teacher 
survey) were dropped from further consideration because their reliabilities fell below a threshold of 0.25. 
Information from the teacher surveys was then aggregated to the school level and then merged with 
principal survey responses as well as demographic and achievement data from EdSource and the 
California Department of Education to create the final school-level analysis file. 

The next step in the analysis, using the school-level file, was to combine the reliable teacher and principal 
survey items to create composite variables (scales) representing school qualities, policies, or practices 
potentially related to academic success. Within the project, these were referred to as "sub-domains," 
organized into eight "domains." An in-depth analysis of the sub-domains was performed to ensure the 
included items were positively correlated with achievement and that the groups of items showed adequate 
internal-consistency reliability. This was done by first calculating the zero-order and partial correlations 
of each item on the primary outcome of interest (the API) as well as on the individual English Language 
Arts and Mathematics California Standards Tests (CSTs). Next, the internal consistency of the set of 
items in each sub-domain was checked by evaluating Cronbach’s alpha and checking the dimensionality 
of each set using factor analytic techniques. 1 

Sub-domains were constructed so as to be mutually exclusive; no item was included in more than one 
sub-domain. While some sub-domains included only a single item, some items were not included in any 
sub-domain and were dropped from further consideration. Typically, items were dropped because review 
of the item text or review of response patterns indicated that the items had not functioned as intended. 2 A 
very small number of items were dropped because they appeared completely redundant (i.e., were highly 
correlated) with other items. As with the design of the teacher and principal surveys, the creation of 
domains and sub-domains was guided by previous theory and research. With one exception, separate 
sub-domains were created from teacher survey items versus solely from principal survey items; in only 
one case were teacher and principal responses combined in a single sub-domain. (This combined sub- 
domain was based on teacher and principal responses to the identical question, "Our school uses a 
standards-based report card.") 

Initially, the 2004 "Growth" API was the primary focus of analyses. When the 2005 "Growth" API was 
released, analyses were rerun using the 2005 test data and 2005 demographic control variables. Results 
were essentially unchanged; the 2005 results are the primary reporting focus. 

As mentioned previously, a "Growth" API is a cross-sectional measure based on data from a single year. 
The term "Growth" indicates that it is calculated in the same manner as the preceding year's "Base" API. 
Thus, the arithmetic difference between current-year "Growth" and prior-year "Base" APIs represents the 
change in a school's performance from one year to the next. Only cross-sectional measures, not change 



1 Indeed, as a result of the factor analysis some of the original sub-domains were split up into smaller ones. 

2 This was done by first evaluating the internal consistency of the set of items in each sub-domain using Cronbach’s alpha and 
then checking the dimensionality of each set using factor analytic techniques. 
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measures, are used in the analyses for the present study. In addition, an outcome variable based on 
schools' average API between 2002 and 2004, with greater weight on more recent years, was examined. 
This "API Composite" gave results similar to, but generally weaker than, those for recent single-year API 
outcome variables. 

Results were also examined for school-level variables created by averaging CST scores in 
English/Language Arts across all K-5 grades within the school, and similarly for Mathematics. Analyses 
of these separate outcome measures showed little difference in variables predictive of performance in one 
subject area or the other. 

As mentioned above, the sub-domains were organized into eight categories (domains) representing areas 
of focus such as the school's instructional program, parental involvement, and use of achievement data. 
For the primary regression analyses, we first regressed the outcome (e.g., 2005 Growth API) on all sub- 
domains within a given domain, together with a set of demographic variables chosen to control for 
residual effects of socioeconomic status and student characteristics that were still present within the 
narrow SCI band specified in framing the study population. The primary statistic of interest in examining 
these regressions was the percent of variance accounted for by the sub-domain composite variables 
(adjusted R 2 ). 3 

For each domain, we next fit a parsimonious regression model that included school demographic 
characteristics and only a subset of composite measures of teacher and principal responses that 
collectively accounted for a substantial portion of the variation in API scores that was captured using the 
entire domain. We then used the estimates from this regression model to predict the API for each school, 
but assuming each school had demographic characteristics equal to the average of the schools in the 
sample. The extent to which these predicted API scores vary across schools gives us a measure of the 
extent to which the composite variables from this domain account for variation in API scores. We report 
the difference in predicted API scores between schools at the 75 th and 25 th percentiles of the distribution 
of predicted scores. This difference can be interpreted as the difference in API scores we would expect to 
observe between two schools (with identical demographic characteristics), one of which scored at the 75 th 
percentile on the combined level of the composite variables in the domain, and the other of which scored 
at the 25 th percentile. A final analysis employed the same procedures, but included the final subsets of 
sub-domains from all eight domains in a single model. 

All figures giving percent of variance explained are adjusted for the number of predictors in the model. 
This adjustment facilitates meaningful comparisons among models with different numbers of predictors. 
However, in cases where the sub-domains not included in the final model contribute very little to the 
prediction, the adjustment can occasionally result in a higher percent of variance explained by the sub- 
domains in the final model than in the full set of sub-domains. 



3 This is measured as the difference in explained variance between the full model containing both the sub-domain and 
demographic variables and the baseline model containing only the demographic control variables. 
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Magnitude of API Differences 
(holds demographics constant at sample mean) 


Domain 


Standard Deviation 
of Predicted API 
Distribution 


Interquartile 
Range of Predicted 
API Distribution 


Implementing a Coherent, Standards-Based 
Instructional Program 


17.6 


22.0 


Ensuring Availability of Instructional Resources 


16.9 


22.4 


Using Assessment Data to Improve Student 
Achievement and Instruction 


16.7 


22.1 


Prioritizing Student Achievement 


16.3 


22.6 


Encouraging Teacher Collaboration and Professional 
Development 


11.0 


14.2 


Enforcing High Expectations for Student Behavior 


12.3 


16.6 


Involving and Supporting Parents 


9.9 


14.4 
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APPENDIX C 



Descriptive Statistics of the Schools in Our Sample and Their Student 
Populations 4 



Overall Demographics and Performance of Participating Schools 

In 2003, when the study’s sample was derived, 547 non-charter elementary schools were in the 25 th -35 th 
percentile of the school characteristics index (SCI). Out of the 269 schools identified from this group for 
the study’s sample, 257 agreed to participate in the study and returned the principal’s survey and at least 
some teacher surveys, with the overwhelming majority returning at least 80% of the teacher surveys. 
These 257 schools are referred to below as “participating” or “sample” schools. The other 290 schools 
that make up the rest of the 547 non-charter elementary schools in the 25 th -35 th percentile band of the SCI 
are referred to below as “non-participating” or “non-sample” schools. 

Schools in 145 school districts participated in the study. Fifty-six of these districts had more than one 
school in the study. According to 2004-05 data, the average school enrollment was 588 students. Of the 
257 participating schools: 

• 33% were situated in elementary school districts and the remaining 67% in unified school districts. 

• 15% of the schools operate on a year-round calendar. 

• 98% received Title I funding and 34% participated in Program Improvement— meaning that these 
Title I schools had not met the federal Adequate Yearly Progress benchmark for at least two consecutive 
years. 



Student Demographics 

The student population attending the participating schools was diverse and facing particular challenges. 
The median value of a few variables illustrate the challenges. (The median is found by arranging schools 
in order of their values on a particular variable and finding the value in the middle of the group. As a 
result, there is no one school that represents the median on all of these factors.) At the median sample 
school, 40% did not speak English as their primary language and 78% participated in the free and 
reduced-price meals program for low-income families. Among all sample schools, the percent of English 
learners ranged from 1% to 80%, and the percent of students participating in the free and reduced-price 
meals program ranged from 17% to 100%. 

The median percentage of students by ethnicity at the sample schools was as follows: 68% Hispanic, 14% 
white, 4% African American, and 3% Asian. The median value for the combination of American Indian, 
Filipino, Pacific Islander, and multiethnic students, along with those who did not indicate an ethnicity, 
was 4%. In most but not all schools the majority population was Hispanic. The highest percentage of 
students of a given ethnicity found in any one school was: 99% Hispanic, 94% African American, 54% 
white, 42% Asian, 38% American Indian, 31% Filipino, and 10% Pacific Islander. Conversely, the lowest 
percentage for most ethnic groups was 0%, but all schools had at least 1% Hispanic students. 

In the median sample school, 32% of students had parents who were not high school graduates. The 
median values for other parental education levels were 33% high school graduate, 21% some college, and 
11% completed college or graduate school. 



4 Data used in this section is from 2004-05 unless otherwise noted. 



1 • Similar Students, Different Results • EdSource 




School Performance 

The 2005 Growth API for the average school participating in the study was 702. The lowest-performing 
school scored 569, and the highest performing school scored 821. 

The state also assigns a rank from one to 10 to each school in the state at the start of each API cycle, with 
a 10 identifying the 10% of schools that are the highest performing. For the 2004 Base API, none of the 
schools in the study was ranked a 10. Only 4% ranked at seven or higher, meaning within the top 40% of 
elementary schools. Conversely, 7% were ranked at one, the bottom decile. The remaining 89% of 
schools ranked from two to six, with the majority a three or four. 



Comparing sample schools to non-participants and to all California 
elementary schools 

School Characteristics 

Participating, non-participating, and all elementary schools had a similar percentage of schools in unified 
and elementary districts. 

The average enrollment in our sample schools was slightly smaller (588 students) than that of non- 
participating schools (630) and slightly larger than the average California elementary school (567). Class 
sizes, however, were similar for all three categories. While only 15% of the sample schools operated 
year-round, more than twice as large a proportion of the non-participating schools (31%) did. This is 
compared to 20% of all California elementary schools. 

Regarding geographic representation, the participating schools were fairly representative of elementary 
schools statewide, but not as representative of schools in the 25 th -35 th percentile of the SCI. The 
percentages of schools from each of three regions— northern, southern, and central California— were 
about the same among schools participating in the study and those statewide. However, considering just 
the SCI band, schools in northern and central California were somewhat over-represented among 
participants, and those in southern California were somewhat under-represented. The table below gives 
the exact percentages. 



Percent of Schools by Region 



Region 


Sample 

Schools 


Non-Sample 

Schools 


All California 
Elementary 
Schools 


Northern 

(Santa Cruz, Santa Clara, Merced, 
Madera, Mono, and all other counties 
northward) 


34% 


29% 


38% 


Central 

(areas between designated northern and 
southern areas) 


13% 


6% 


8% 


Southern California 

(San Luis Obispo, Kern, San Bernardino, 
and all other counties southward) 


53% 


65% 


54% 



Student Demographics 

Sample schools and non-participating schools had similar ethnic compositions as well as similar 
percentages of English learners and students from low-income families. 

As the table below shows, the median percentages of student ethnicities among sample and non-sample 
schools paralleled each other quite closely. One small exception can be found in the percentage of white 
students, with the median sample school having 14% as opposed to 10% in the median non-sample 
school. On the other hand, the sample schools looked somewhat different than elementary schools 
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statewide. In particular, the median sample school had a considerably smaller percentage of white 
students and larger percentage of Hispanic students than the median school statewide. 

Ethnic Composition of Participating, Non-participating, and all Elementary Schools 



Ethnicity 


Median Values for: 


Sample Schools 


Non-Sample 

Schools 


All California 
Elementary 
Schools 


African American 


4% 


5% 


3% 


American Indian 


0.4% 


0.4% 


0.4% 


Asian 


3% 


2% 


3% 


Filipino 


1% 


1% 


1% 


Hispanic 


68% 


68% 


41% 


Pacific Islander 


0.3% 


0.4% 


0.3% 


White 


14% 


10% 


28% 



In terms of English learners and students from low-income families, sample schools were very similar to 
non-participating schools but quite different from the statewide set of elementary schools. The English 
learner population was 40% in the median participating school, 41% in the median non-participating 
school in the band, but only 24% in the median school statewide. Similarly, for the free and reduced-price 
meals program, the median participating and non-participating schools both had 78% of students in the 
program, whereas only 58% of students from all California elementary schools were in it. 

Regarding parental education levels, the study’s schools and non-participating schools had a very similar 
proportion of students with parents who did not have a college degree. In the sampled schools, the median 
was 89% while in the non-participating schools, the median was 88%. Statewide, the median was 78%. 



School Performance 

Participating schools had slightly higher 2005 Growth API scores on average than non-participating 
schools in the band, as indicated in the table below. As one would expect, the API scores among all 
California elementary schools had a higher maximum, higher average, and lower minimum than the 
scores of the participating and non-participating schools. 



2005 Growth API Summary 





Sample 

Schools 


Non-Sample 

Schools 


All California 
Elementary 
Schools 


Maximum 


821 


848 


999 


Average 


702 


696 


753 


Minimum 


569 


593 


317 
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Subscription to EdSource Materials 

EdSource produces publications on California education policy, and the largest subscriber group to these 
publications is school district offices. Among the 547 schools in the 25 th -35“ percentile of the SCI, 235 
come from districts that subscribe to EdSource materials and 312 do not. Fifty-four percent of the 
“subscriber” schools, and 41% of the “non-subscriber” schools participated in the study. Capitalizing on 
the established relationship between EdSource and the subscriber schools, we deliberately over-sampled 
subscribers to maximize the participation rate. However, we adjusted for the fact that we had a higher 
percentage of subscribers in our sample through the use of sampling weights. 



Within our Sample, Comparing the Highest- and Lowest- performing Schools 
to the Sample’s Mean/Median on a Variety of Variables 

Another way to look at the demographics is by high- and low-performing schools. For this purpose, high- 
performing schools are those that scored one standard deviation or more above the mean for our sample 
on the 2005 Growth API. Low-performing schools scored one standard deviation or more below the 
mean. The following data represent the 38 high-performing and 41 low-performing schools identified in 
this way. 

At the high-performing schools, slightly fewer parents lacked a high school diploma and slightly more 
had a college degree, as the table below indicates. 



Parental Education at the Median School (2003-04) 



Parental Education Level 


In High-performing 
Schools 


In Low-performing 
Schools 


Not High School Graduate 


29% 


34% 


High School Graduate 


36% 


34% 


Some College 


25% 


19% 


College Graduate 


9% 


6% 


Graduate School 


3% 


3% 



Data: California Department of Education - CBEDS and API Files 



Student Demographics in 2004-05 in the Sample, High-performing Schools, and Low-performing 
Schools 



Variable 


Median value for: 


All Schools in 
Sample 


High-performing 

Schools 


Low-performing 

Schools 


% African American 


4% 


6% 


4% 


% Asian 


3% 


3% 


2% 


% Hispanic 


68% 


58% 


70% 


% White 


14% 


15% 


15% 


% English Learner 


40% 


33% 


49% 


% Free/Reduced-Price Meals 


78% 


71% 


82% 


% Parent Not High School 
Graduate* 


32% 


29% 


34% 



*Datafor 2003-04 
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2004 Base API Scores for the Following Subgroups in all Schools in the Sample, High-performing 
Schools, and Low-performing Schools. 



Subgroup 


Median 2004 Base API score for: 


All schools in 
Sample 


High-performing 

Schools 


Low-performing 

Schools 


African American 


655 


722 


608 


Asian 


710 


866 


655 


Hispanic 


665 


720 


603 


White 


734 


790 


697 


Socioeconomically 

disadvantaged 


665 


721 


613 



Summary 

While the schools analyzed for this study may have differed from one another in terms of specific poverty 
levels and other factors, they all shared a relatively high level of challenge compared to other elementary 
schools in the state. In addition, the sample schools were generally representative of the set of schools of 
interest— those in the 25 th -35 th percentile of the school characteristics index. While somewhat different in 
some ways such as geographic representation, the sample schools and non-participating schools from the 
SCI band were very similar in key factors such as ethnic composition, the percentage of English learners 
and low-income students, and parental education. 



Appendices A-C from: Williams, T., Kirst, M., Haertel, E., et al. (2005). Similar Students, Different Results: Why Do Some 
Schools Do Better? A large-scale survey of California elementary schools serving low-income students. Mountain View, CA: 
EdSource. 



Copyright 2005 EdSource, Inc. All rights reserved. Call 650/917-9481 for permission to reprint. 
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